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1. Introduction 


The NASA Langley Research Center (LaRC) and the Department of 
Transportation/Federal Aviation Administration (DOT/FAA) are conducting 
collaborative research activities to ensure effective technology development and 
implementation of regulatory and design guidance to support the use of Synthetic Vision 
Systems (SVS) and Enhanced Flight Vision Systems (EFVS) advanced cockpit vision 
technologies in Next Generation Air Transportation System (NextGen) operations. 

SVS/EFVS technologies have the potential to provide an additional margin of safety and 
performance to enable the implementation of operational improvements for low visibility 
surface, arrival, and departure operations with equivalent efficiency as visual operations. 

SVS uses a terrain and obstacle database to present a computer-generated view of the 
outside world, often on a Head-Down Display (HDD). While HDDs are cheaper and 
more widely available than Head-Up Displays (HUDs), presentation of SVS information 
on a HDD display has the potential to significantly increase pilot head down time which 
may detract from ground-referenced navigation and hazard detection/avoidance. 

EFVS uses active or passive sensors to present a visual image of the outside view on a 
HUD. Previous research on HUD use in airborne applications suggests that clutter and 
masking can lead to decreased conflict detection. Safe and effective use of EFVS 
information presented on a HUD will require assessment of the impact of EFVS 
presentation on ground conflict detection and avoidance. 

In this report, the research literature is reviewed to assess awareness and detection of 
traffic and obstacles when using SVS and EFVS systems, as follows: 

• Qualify the critical issues influencing the time required, accuracy, and pilot 
workload associated with recognizing and reacting to potential collisions or 
conflicts with other aircraft, vehicles and obstructions related to the use of SVS 
and EFVS (SEVS) technologies during approach, landing, and surface operations. 

o This work includes the effect of HDD and HUD implementations of SVS 
and EFVS, respectively, as well as the influence of single and dual pilot 
operations. 

• Identify methods and strategies of adding Cockpit Display of Traffic Information 
(CDTI) with head-down SVS and head-up EFVS in low- visibility landing and 
surface operations and their effect on time required, accuracy, and pilot workload 
for recognizing and reacting to potential ground collisions or conflicts. This work 
includes the impact of emerging requirements for CDTI and alerting for NextGen. 

Based on this review, a knowledge gap assessment is conducted to create 
recommendations for subsequent ground and flight testing which promote the safe and 
effective implementation of SEVS technologies for NextGen. 
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2. Background 

Synthetic vision (SV) is a computer-generated image of the external scene topography 
that is generated from aircraft attitude, high-precision navigation, and data of the terrain, 
obstacles, cultural features, and other required flight information (Figure 1). The term, 
SVS, as opposed to SV, is often used to indicate that this database information is 
enhanced with integrity monitoring to ensure the validity of the databases and/or 
independent navigation accuracy verification. Part of this integrity monitoring function 
may also include real-time traffic surveillance and obstacle/object detection (Parrish et al, 
2008). 

Under NASA’s Aviation Safety Program/Synthetic Vision Project, NASA and its 
partners in industry and the FAA, developed and deployed SVS technologies for 
commercial, business, and general aviation (GA) aircraft. SVS concepts were shown to 
provide significant improvements in terrain awareness and reductions for the potential of 
Controlled-Flight-Into-Terrain incidents/accidents compared to current generation 
cockpit technologies (Kramer, 2004; Kramer, 2005; Arthur 2003; Parrish et al, 2008). 

Enhanced Vision (EV) is an electronic means to provide a display of the external scene 
by use of an imaging sensor, such as a Forward-Looking InffaRed (FLIR) or millimeter 
wave radar. EV has been used for decades by the military for night missions, especially 
for low-level navigation and targeting. Various development activities and 
demonstrations have been conducted. The FAA conducted its own EV technology 
demonstration program - then called a Synthetic Vision System - to assess the state-of- 
the-art and the operational implications of the technology (Burgess, 1993). As HUD 
technology has migrated into commercial and business aircraft, EV technology and its 
potential for all-weather operations and improved safety naturally followed (e.g., see 
Looking Ahead, 1993). EV has been shown to be complementary to SV due to their 
inherently different technological bases (Bailey, 2008). 

The applications of EV technology for commercial, business, and GA aircraft were 
energized in January 2004 (FAA, 2004a) when Title 14 of the US Code of Federal 
Regulations (CFR) §91.175 was amended such that operators conducting straight-in 
instrument approach procedures (in other than Category II or Category III operations) 
may now operate below the published Decision Altitude (DA)/Decision Height (DH) or 
Minimum Descent Altitude (MDA) when using an approved EFVS. An EFVS, in this 
application, is an integrated conformal display of EV and symbology shown on the pilot’s 
HUD (see Figure 2). The required conformal symbology includes the flight path vector, 
flight path angle reference, and flight guidance with other non-conformal flight 
information shown as necessary and appropriate. The EFVS is approved for use in lieu 
of natural vision. This rule change provides “operational credit” for EV equipage. No 
such credit exists for SV. 

Minimum Aviation System Performance Standards (MASPS) have been published as 
DO-315 (RTCA, 2008), which provides the system performance standards for EFVS 
(“operational credit”) and SV (“no operational credit”). 
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Figure 1 : Honeywell Synthetic Vision Concept 


2.1. SV and EFVS Operations 

Even though SV and EFVS are certified and operational, research issues still abound, 
especially as the applications for these technologies are broadened and expanded. 

SEVS technologies have been identified as fundamental building blocks to NextGen 
(FAA, 2004b); integral to the concept of “equivalent visual operations.” Equivalent 
visual operations imply the ability to achieve or even improve on the safety of current- 
day Visual Flight Rules (VFR) operations, maintain the operational tempos of VFR, and 
even, perhaps, retain VFR-derived procedures independent of the actual weather and 
visibility conditions. These vision technologies (SEVS) serve as enabling technologies to 
meet that challenge following the precedent that SEVS may be used in lieu of a pilot’s 
natural vision. This is currently only the case for EFVS under §91.175 (1) and (m). 
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Figure 2: Enhanced Flight Vision System Concept 

In this report, a truly “equivalent visual operation” is not assumed. While this is the 
ultimate objective of using this technology, this initial work examines SEVS technologies 
only to enable or assist in low visibility approach and landing, surface, and take- 
off/departure operations where there is no change in the present-day roles and 
responsibilities of Air Navigation Service Providers (ANSPs) for spacing and separation. 
Future growth and usage of SV and EV “vision systems” technology may eventually 
enable the flight crew to perform VFR-like operational capability (i.e., “see-to-follow,” 
“see-and-avoid,” and “own navigation” including the pilot’s ability and responsibility to 
self-separate from other traffic, terrain, and obstacles) but in this report, the implications 
of creating these capabilities are not considered. Future work should investigate the 
vision system technology (SEVS) requirements for an equivalent visual operational 
capability (R-l). 

In this work, the current and incremental improvements in operational capabilities using 
SVS and EFVS are considered as follows: 

• SEVS technologies are used under VFR and Instrument Flight Rules operations, 
day and night. 

• In addition to the current “no operational credit” use of SV, the use of SV is 
examined to enable descent below the published DA or DH to 100 ft height above 
the touchdown zone elevation (TDZE) from a straight-in instrument approach 
procedures with published vertical guidance. 
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• In addition to the current “operational credit” use of EFVS identified under 14 
CFR §91.175 (1) and (m), the use of EFVS is examined to enable descent below 
the published DA or DH, landing and roll-out in visibility as low as 1000 ft 
Runway Visual Range (RVR) flying a straight-in instrument approach procedure 
with published vertical guidance. 

• Further, the use of EFVS is assumed to enable take-off and surface operations in 
visibility conditions down to 700 ft RVR. 

2.2. SEVS Research Issues and Report Organization 

Two overarching research issues are addressed in this report arising from present and 
future operational concerns associated with SEVS technology: 

1) SV operations are presently focused on HDD implementations since they are less 
expensive and have a broader user base (i.e., it is not practical to install a HUD in 
many aircraft). Presentation of SV information on a HDD during low altitude and 
surface operations has the potential to significantly increase pilot head-down time 
which may detract from ground-referenced navigation and hazard 

detection/ avoidance . 

2) In EFVS operations, the HUD display must present all EFVS-required imagery 
and symbology and for compliance with 14 CFR §25.1301 and §25.1303, data 
elements of primary flight displays which are essential or critical to the phase of 
flight. Conversely, for compliance with 14 CFR §25.773, HUD symbology must 
not excessively interfere with a pilots’ forward view and their ability to visually 
maneuver the airplane, acquire opposing traffic, and see the runway environment. 
These conflicting requirements necessitate an examination of how EFVS 
information presented on a HUD impacts its safe and effective use and especially, 
the potential for HUD clutter and obscuration to negatively impact traffic and 
obstacle collision detection and avoidance. 

In both cases, a critical issue is the ability of a human observer to perform “target 
detection.” In Section 3, a review of traffic and obstacle detection by a human observer 
is presented. This work points to the wealth of data pertaining to this activity, and more 
importantly, highlights that the speed and accuracy of traffic and obstacle detection is a 
function of awareness (i.e., is the observer alerted to the conflict?), workload (i.e., is the 
observer engaged in a divided attention or full attention task?), localization (i.e., does the 
user have other information directing them where to look?), and the characteristics of the 
object to be seen, the device being used to image and display the object, and the 
atmospheric properties, lighting, and background in which the object/traffic is being 
viewed. 

Research is then reviewed, in Section 4, regarding pilot/flight crew tasking for awareness 
and detection of targets/obstacles. Head-down to head-up transitions and pilot/crew roles 
and responsibilities are presented. This work discusses the issues of transitioning from 
head-down to head-up operations and emphasizes how this impacts target (i.e., traffic or 
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obstacle/object) detection, recognition, and resolution while attending to required flight 
deck and piloting tasks. 

To address the specific nuances of the second issue, the state-of-the-art regarding clutter, 
obscuration (masking), and attention capture using HUDs is reviewed in Section 5. 

These issues have garnered a wealth of study and merit special consideration as to its 
direct applicability in this research. This work is also related to target (i.e., traffic or 
obstacle/object) detection, recognition, and resolution while attending to required flight 
deck and piloting tasks. 

In Section 6, these two topics are considered as they might be influenced and affected by 
the design and use of CDTI for approach and landing, surface, and take-off/departure 
operations. This work includes the impact of emerging technology for traffic indication, 
warning, and alerting during approach, landing, and surface operations. 

Finally, in Section 7, concluding remarks and recommendations are presented. This 
section summarizes the critical research areas that need to be explored further to advance 
the state-of-the-art and to effectively implement and possibly expand the operational 
capabilities of SEVS technologies forNextGen. 

2.3. Assumptions and Limitations 

Several assumptions and limitations have been invoked for this work. 

First, the limitations: 

1 . This work is not intended to be stand-alone; instead, numerous references are 
used to point the reader where additional, important information is contained. 

The intent of this work is to integrate numerous disparate pieces of information 
from which to form a unifying picture as well as to identify knowledge gaps 
which highlight future research requirements for SEVS technologies. 

2. This work focuses on the use of CDTI and the pilot’s awareness and detection of 
traffic and obstacles almost exclusively on the runway and airport surface. This 
emphasis examines traffic detection issues in those operations (i.e., during final 
approach and landing, taxi, and departure/take-off) where SEVS technologies 
provide notable operational benefits. Significant work has been conducted in 
other flight phases (e.g., en-route merging and spacing) and this work may be 
referenced as relevant, but it is not extensively reviewed. 

3. The work considers only fixed-wing aircraft operations. The analyses may be 
applicable to vertical take-off and landing or rotary wing vehicles, but this work 
has not been validated for the unique operational aspects of these vehicles. No 
warranty as to the applicability of this review should be assumed or implied. 

4. This work assumes that the imaging sensor for the EF VS is a FLIR. This 
limitation may not be a serious limitation since only FLIR sensors have been 
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approved to date as an EFYS, but it does merit note as a slight loss in generality is 
created. 

The primary assumption imposed in this review is that the pilots (and crew) are 
sufficiently trained in the use and operation of the equipment. Also, that during crewed 
operations, both pilots are effectively functioning as a crew and a break-down in crew 
coordination or crew resource management has not occurred. 

At present, the roles and responsibilities of the ANSPs for spacing and separation from 
other traffic are assumed to be unchanged by the presence or use of these flight deck 
technologies. These technologies are not intended to change the roles and responsibilities 
of the airport services providers for foreign object damage control, wildlife control, or 
other airport infrastructure and service functions. Future work should explore the impact, 
if any, of SEVS in changing the roles and responsibilities of the ANSP (R-2). 

SEYS technologies also offer unique capabilities which might be used in lieu of certain 
airport and aircraft infrastructure or equipment for low visibility approach, landing, 
surface, and take-off/departure operations. Future work should explore the potential for 
SEVS to replace the need for or functional elements currently required for Surface 
Movement Guidance and Control Systems and for approach lighting systems (R-3). 

SEVS technologies may eventually enable the pilot/crew to perform “see-to-follow,” 
“see-and-avoid,” and “self-navigation” including the ability and responsibility of the 
pilot/crew to self-separate from other traffic, terrain, and obstacles. As previously 
mentioned, this work does not focus on these new capabilities but the work is 
fundamentally applicable to it. 
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3. Traffic and Obstacle Detection 


The visual traffic and obstacle detection task is, in essence, a human observer performing 
“target acquisition” of which a wealth of data and research are available, mostly in the 
military domain. These works are too numerous to detail; instead, key findings are 
extracted from this work and briefly summarized to establish “first-principles.” From 
this basis, later sections extend these principles into research specific to SEVS operations 
and display design issues to qualify the time required, pilot workload, and accuracy of 
traffic and obstacle detection and the attendant critical issues. 

Two perceptual mechanisms are considered in this research: 

• Target detection by a human observer looking through cockpit windows; 

• Target detection by a human observer looking at cockpit displays, using a 
HUD or HDD. 

3.1. Target Detection by a Human Observer 

While a wealth of research and data are available, the target detection task by a human 
observer is complicated by many complex and incompletely understood processes and 
interactions, especially when applied to the flight environment. 

Target detection research (Boff and Lincoln, 1988) shows that the probability that a target 
will be recognized by a human observer is a function of: 

• the probability that the observer is searching an area that is known to 
contain a target, looks with his/her foveal vision for a specified glimpse 
time (54 s) in the direction of the target; and, 

• the probability that the displayed target image is viewed foveally for one 
glimpse period with sufficient contrast and size to be detected; 

The later element - the saliency of the object in terms of whether the target is of 
sufficient contrast and size - can be considered as a necessary but perhaps not sufficient 
prerequisite for successful target detection by a human observer. 

3.2. Target Perception By A Pilot With and Without EV 

The concept of target saliency and the use of SEVS technologies for target detection by a 
pilot is schematically diagramed in Figure 3. The diagram does not attempt to map out a 
cognitive process, but rather, tries to capture the physical effects and perceptual 
mechanisms involved in determining if the necessary but not necessarily sufficient 
condition for visual target acquisition exists (i.e., whether the human’s perception of the 
target has sufficient contrast and size). 



Figure 3: Target Detection Diagram 


3.2.1. Source 


The perceived target contrast and size is obviously driven by the contrast and size of the 
“source information.” In Figure 3, this source information, shown in a bottom-up 
paradigm, drives the perceptual paths (i.e., “Enhanced Vision” or “Natural Vision”) 

The target angular size and the apparent target-background luminance ratio (contrast) of 
the “source information” depend primarily on four factors (see Boff and Lincoln, 1988): 

• the target attributes 

• the attributes of the target background 

• the medium through which it is viewed (i.e., the atmospheric properties such as, 
fog, haze, moisture content, etc.) 

• and the characteristics that illuminate the target, background, and atmosphere 
(i.e., sun angle, moon phase, etc.). 

3.2.2. Detection Mechanisms 


This “source information” flows into two possible perception mechanisms: 
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1 . Using Enhanced Vision 

2. Using Natural Vision (i.e., the out-the-window (OTW) view) 

(The use of SV or CDTI is not considered in this process since this section is concerned 
with unaided visual target detection by a human observer.) 

The perception of the target - whether the target is of sufficient size and contrast - is thus 
affected by how this source information (i.e., the target-background-medium attributes) is 
modified along these paths before it is presented to the human observer. 

In the case of EV, these modifications include how this source information is “perceived” 
by the EV sensor and what image transformations or distortions are introduced in the 
process. Present-day FLIRs transform the radiated thermal energy into gray-scale level 
(monochromatic) imagery; that is, the thermal signature. 

The first-principle involves the sensor detector characteristics including its wave-length, 
resolution, sensitivity, noise, and image processing characteristics. Target detection is 
critically dependent on how these image properties are “matched” to the imaging sensor 
and the characteristics of the environment in which the imaging occurs. For EFVS 
applications, the FLIR sensor typically provides a wide bandwidth of detector 
wavelengths, designed to provide a best-case compromise between thermal sensitivity 
and transmissivity through atmospheric moisture content for runway, terrain, and 
airport/approach lighting systems. 

Analytical tools have been developed to predict infrared sensor system performance in 
this task. Moderate resolution atmospheric transmission simulations (Beier and 
Gemperlein, 2004) model the atmosphere and can predict the radiative properties for a 
wide range of wavelengths and spectral resolutions, enabling the calculation of 
transmittance and radiance from a target. It uses different climate models, geographical 
latitudes, time of year (i.e., seasons influencing, for example, sun angle and prevailing 
atmospheric temperature) and with each climate model, different aerosol types. 

However, the conditions of the atmosphere alone aren’t enough to predict performance. 
The size of the target and the radiated/reflective/emitted differences against the 
background as well as the spatial resolution and sensitivity of the optics and the detector, 
and the noise of the detector and signal processing alter the target prominence against the 
background. 


3.2.3. Detection Avenues 

Visual target detection is shown through any of four possible avenues: 

1 . Using EV on a HDD 

2. Using EV on a HUD 

3. Using the OTW view, through a HUD 

4. Using the OTW view 

These four avenues subtend different fields-of-regard/fields-of-view (FOV) and also, 
invoke different transformational processes of the source information to the pilot 
perception of the image. For instance: 
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• Using the OTW view : Target detection using natural vision OTW occurs by the 
pilot/crew looking through the aircraft windows. The field-of-regard for which 
this detection is possible is the extent of the windows and the range of head/eye 
movement available by the crew to look around any aircraft structure. Guidance 
for this field-of-regard (i.e., window size) for target detection (i.e., collision 
avoidance) is given under Advisory Circular (AC) 25-773. In this detection task, 
the “source image” properties are modified by the optical characteristics of the 
windscreen. Although the goal in the design and development of the windscreen 
optical characteristics is to provide no modification, perfection is not possible. 
Under 14 CFR §23.773, “The windshield and side windows forward of the pilot's 
back when the pilot is seated in the normal flight position must have a luminous 
transmittance value of not less than 70 percent. 41 Testing has shown that these 
optical performance requirements has generally been achieved and maintained 
through an aircraft’s service life (Quinn et al, 1996). The influence of canopy 
distortion in a target detection task for civil transport aircraft should not be a 
factor, unlike military/fighter aircraft (Task and Goodyear, 1999). 

• Using the OTW view, through a HUD : Target detection using natural vision 
OTW can also occur by the pilot/crew looking through the HUD and aircraft 
windows. In this case, the target must be located within the HUD field-of-regard. 
In this detection task, the “source image” properties are modified by the optical 
characteristics of the windscreen as well as the optical characteristics of the HUD 
(i.e., transmissivity) and any symbology or imagery that may obscure this 
detection. SAE Aeronautical Standard AS-5088, “Minimum Performance 
Standard for Airborne Head Up Display (HUD)” specifies that the light 
transmissivity of the HUD combiner shall be greater than 70%. This degradation 
is in addition to possible canopy transmissivity losses, resulting in some potential 
visual performance degradation. The issues of obscuration due to symbology and 
attentional influence of HUD operations are discussed in following sections. 

• Using EV on a HUD: Target detection using EV can occur by the pilot/crew 
looking at the HUD. EV shown on a HUD is, by EFVS definition, conformally 
drawn (i.e., the field-of-regard of the sensor and the HUD are matched and the 
imagery overlays the real world as seen through the HUD combiner); thus, for a 
target to be detected using an EV-HUD, the target must be located within the 
HUD FOV. 

• Using EV on a HDD : For the EV detection task shown on the HDD, the field-of- 
regard of the EV sensor and whether it is fixed or slewable to the aircraft affect 
whether the target is visible to the EV sensor. Also, the effects from minification 
and magnification of the imagery (i.e., non-conformal display) must be 
considered in the detection task. Both the display as well as the sensor, dictate the 
image resolution presented to the human for the target detection task. In addition 
to resolution, the magnification and minification of the information affects human 
performance. 
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The detection of targets and obstacles using EY imagery critically depends on the display 
device characteristics. The distinction between head-up and head-down displays is not 
only important for their spatial location but also due to their optical characteristics. 

• HDDs afford much greater resolution, more gray scales, luminance, and contrast 
sensitivity than HUDs. HUD gray shades, defined as an increase in brightness by 
a factor of 1.41 (square root of 2), are typically no greater than 7 and rapidly 
reduce as the ambient background luminance increases toward 10,000 ft-Lamberts 
(Karim, 1992). This loss of apparent contrast degrades human performance 
(Lloyd and Reinhart, 1993). 6 shades of gray are typically required for imagery 
(i.e., video) (Rash, 1999). HDDs do not have to contend with a variable visual 
background, they employ much higher luminance and contrast optical properties 
and can use glareshields and other optics to shade the displays from direct and 
diffuse sun light effects (Karim, 1992). 

• Chromaticity can be put into effect on HDDs unlike monochromatic HUDs (e.g., 
see Scribner, Warren, and Schuler, 2000). Typically, EV imagery is 
monochromatic, but the HDD may highlight data aspects using false color cueing 
(e.g., see Toet, 2002). 

3.3. Johnson’s Criteria 

Numerous and sometimes elaborate analyses have been developed to predict human 
detection performance, essentially quantifying whether the perceived target is of 
sufficient contrast and size (e.g., to name just a few: Aviram and Rotman, 2000 and 2001, 
Vollmerhausen, Jacobs, and Driggers, 2004; Watson, Ramirez, and Salud, 2009; see also 
Boff and Lincoln, 1988). 

Instead of using these models, a rough-order-of-magnitude criterion, for comparative 
analysis only, is used here. Johnson’s criteria attempts to quantify minimum resolution 
requirements for human awareness, detection, recognition and identification of traffic or 
objects/obstacles. 

A series of experiments (Ratches et al, 1997) were conducted to quantify target detection 
probability as a function of the resolution of the sensor/display system (i.e., the target 
angular size). This work showed that, whether using an unaided eye or other sensors, 
such as night-vision goggles or thermal imaging systems, four tasks were involved in the 
target acquisition process: 

1) Detection: correctly discriminating an object in the image from background and 

system noise. 

2) Orientation: correctly determining the detected target aspect or direction of 

movement. 

3) Recognition: correctly determining the class membership of the target. For 

example, is it a truck? A tank? etc. 

4) Identification: correctly determining the exact identity of the target, e.g., for 

automobiles, is it a Ford or Chevrolet? 
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These same and similar processes in detection, recognition, and identification for air-to- 
air target acquisition tasks have also been found (e.g., see Rohrer, 1996). 

These experimental data led to the development of Johnson’s criteria for target 
acquisition. Johnson’s work presumed that the ability of the observer to detect, recognize 
and identify a target were a function of how well a critical dimension of the target could 
be resolved by the sensor-display system. An example of the data is shown in Figure 4, 
indicating the probability of detection and recognition (of a man or a vehicle) is a 
function of the number of sensor-display cycles (sensor/display line pairs or roughly 
pixels) depicting the object. 



H - CrCLES CRITICAL TARGET DimEMLlQH 


Figure 4: Probability Of Detection And Recognition As Function Of Resolution Across 
Critical Target Dimensions (from Ratches, 1997). 


Johnson’s work evolved into straight-forward criteria for display and sensor requirements 
for visual acquisition as shown in Table I. The “critical target dimension” was based on 
intuition and was usually chosen to be the minimum dimension. 


Table I - Johnson’s Criteria - Resolution Requirements 


Task 

Line Resolution per Minimum 
Dimension 

Detection 

1.0 ± 0.25 line pairs 

Orientation 

1.4 ± 0.35 line pairs 

Recognition 

4.0 ± 0.8 line pairs 

Identification 

6.4 ± 1.5 line pairs 


These criteria only pertain to the target size of the problem; that is, the angular extent of 
the target and the associated resolution of the display and sensor to generate the image for 
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human visual acquisition. The other components of the visual acquisition task - e.g., the 
apparent target-background contrast - are dependent upon innumerable other factors such 
as atmospheric conditions (clouds, fog, haze, moisture content, etc.), target 
color/material, background illumination and color, sun angle, background (sky/ground), 
etc and sensor/display contrast performance. Even though these are critical issues to the 
task, these are not influenced by the display resolution directly. 

Johnson’s criteria has been shown to be a rough, first-order approximation as numerous 
analyses have shown it to be flawed in several respects (McDonald and Yorst, 2002); 
however, for comparative analyses associated with angular extent and resolution of a 
target, Johnson’s criteria is applied herein to evaluate EY and display sensor resolution 
effects on visual target acquisition. 

3.4. Visual Search and Gaze 

Whether a target, in a visual detection task, is of sufficient contrast and size can be 
considered as a necessary but perhaps not sufficient prerequisite for successful target 
detection by a human observer. However, most of the elements that dictate this 
prerequisite - other than the design characteristics of the EV sensor and display system - 
are not directly a function of the SEVS technologies. They are a function of the task, the 
proximity of the aircraft to the obstacle or traffic, or the weather and time of day. 

The other critical element required for visual target detection is that of the pilot/crew’s 
visual search and gaze. 

Wickens et al (2001) hypothesized that visual gaze was a function of: 

(1) the salience of objects that affect attention capture 

(2) the effort required to move attention, 

(3) the expectancy that information will be obtained; and, 

(4) the value of the information to be found there. 

These characteristics that govern visual gaze are defined by the roles and responsibilities 
of the pilot/flight crew and are possibly influenced by CDTI or the presence of 
traffic/obstacle indications, cautions, or warnings. These elements are to various degrees 
influenced by the design of and technologies on the flight deck, as described in the 
following sections. 
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4. Head-Down and Head-Up Operations 

The crew/pilot tasking and their roles and responsibilities during SEYS operations are 
explored as they impact the pilot’s/crew’s workload, the available time to perform object 
and traffic detection, and their visual scan patterns. 

The detection of traffic and obstacles is not the only tasking of a pilot or crew member. 
Within the operating paradigm of “aviate, navigate, and communicate/ 1 detection of 
traffic and obstacles is certainly one of the tasks contained within the “navigate” element. 
However, the concept of “see-and-avoid” is engrained within all FAA operations under 
both Instrument Flight Rules and VFRs and certainly is of critical importance. “See-and- 
avoid” requires that each person operating an aircraft maintains vigilance at all times. 

During crewed operations, the flight duties of the crew members are dependent upon 
many factors, and tasking can be distributed, with certain crew members having more or 
less responsibility for “see-and-avoid.” For single pilot operations, vigilance for “see- 
and-avoid” must be attended to as part of all required pilot duties. 

In all cases, the cockpit display of traffic information greatly influences the pilot’s ability 
to perform the task of traffic/obstacle awareness and detection. 

4.1. “See-and-avoid” 

The FAA requires under 14 CFR Part 91.113, across all classes of airspace, that “when 
weather conditions permit, regardless of whether an operation is conducted under 
instrument flight rules or visual flight rules, vigilance shall be maintained by each person 
operating an aircraft so as to see-and-avoid other aircraft.” 

Pilots are trained, and through experience, adopt visual scanning procedures both in their 
use of instruments (i.e., a visual scan pattern of the cockpit displays) and in scanning out- 
the -window for other traffic and/or objects (Bellenkes, Wickens, and Kramer, 1997). The 
FAA’s Aeronautical Information Manual (Section 8-1-6) emphasizes that “the time a 
pilot spends on visual tasks inside the cabin should represent no more than % to 'A of the 
scan time outside, or no more than 4 to 5 seconds on the instrument panel for every 16 
seconds outside.” The criticality of outside scanning to meet the see-and-avoid safety 
principle and how to effectively scan for outside traffic is explained in numerous areas 
(see FAA AC 90-48C, “Pilot’s Role In Collision Avoidance”). Unfortunately, studies 
(e.g., Colvin, Dodhia, and Dismukes, 2005) have shown that pilots do not adhere to these 
recommendations and spend considerably more time head-down than head-out. YFR 
pilots in small GA aircraft have been shown to be preoccupied with in-cockpit duties and 
are head-down at least 40% of the time (Sulzer and Skelton, 1976). Significant 
individual differences have been noted (e.g., see Wickens et al, 2001). 

Measured flight performance for aircraft detection is remarkably poor, often casting 
doubt on the effectiveness of the see-and-avoid principle (see Graham and Orr, 1970; 
Andrews, J,W., 1991a, 1991b) In particular, Andrews showed that only 56% of 64 near- 
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collision encounters were sighted by GA pilots during en-route operations. Although 
some studies show an eight to nine times improvement in detection when alerted, others 
have still shown relatively poor performance (Moore, 1998). 

The merit of see-and-avoid operations and the associated human visual, human attention, 
and operational issues have been studied extensively (e.g., see Australian Transport 
Safety Board, 1991; Graham, 1989). Whether see-and-avoid is actually effective or not is 
immaterial. The FAA requires, under 14 CFR Part 91.113, across all classes of airspace 
that see-and-avoid shall be conducted. This work examines the influence of SEVS 
technologies on traffic and obstacle detection which forms part of “see-and-avoid.” It 
should be noted that most of the research cited above for see-and-avoid were almost 
exclusively conducted enroute or outside of the airport terminal maneuvering area. This 
analysis of SEVS technologies focuses instead on low visibility approach, landing, 
departure, and surface operations. 

4.2. Visibility Conditions and Manual Control 

The weather conditions in which the operation is conducted obviously affect the usable 
visual cues outside the aircraft (i.e., OTW visibility). These weather conditions typically 
also affect the visibility of an imaging sensor in an EFVS. 

Less obvious is that the weather conditions also dictate (by FAA regulations) the on- 
board and off-board systems required to conduct the operation. These systems 
significantly alter the degree to which the pilot must “aviate;” that is, how much the pilot 
manually flies the aircraft. These requirements, thus, impact the crew duties and the 
associated workload and attention demands of the pilot during low visibility approach 
and landing operations. The general assumption is that a pilot who is manually flying 
would have less spare attention to devote to target/obstacle detection and avoidance. 

The landing minima classifications of Category I, II, and III are defined as follows: 

• Category (Cat.) I is an instrument approach or approach and landing with a 
decision altitude (height) or minimum descent altitude (height) not lower than 200 
ft and with either a visibility not less than 1/2 statute mile, or a runway visual 
range (RVR) not less than 1 800 ft. 

• Cat. II is an instrument approach or approach and landing with a decision height 
lower than 200 ft but not lower than 100 ft and a runway visual range not less 
than 1200 ft. 

• Cat. Ill is an instrument approach or approach and landing with a decision height 
lower than 100 ft, or no decision height, or a runway visual range less than 1200 
ft. 

Guidance for US approval of low visibility approach, take-off and surface operations are 
contained in AC 120-29 A (“Criteria For Approval Of Category I And Category II 
Weather Minima For Approach”) and 120-28D (“Criteria For Approval Of Category III 
Weather Minima For Takeoff, Landing, And Rollout”). These works do not contain the 
latest regulatory changes associated with EFVS, such as 14 CFR §91.175 (1) and (m), or 
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special authorizations/changes provisioned through FAA Order 8400.13 (‘Procedures for 
the Evaluation and Approval of Facilities for Special Authorization Category I 
Operations and All Category II and III Operations”). These materials show that: 

• Cat. I approaches can be manually flown with reference to raw data or to flight 
director commands or automatically with the autopilot engaged. 

• Cat. II/Cat. Ill approaches are flown using an automatic landing system. For Cat. 
II minima, a flight director, autopilot, or HUD with approved flight guidance is 
used but the landing can be manually- flown. 

• For Cat. Ill minima, an autopilot or HUD with approved flight guidance is used to 
fly the approach, landing, and roll-out. The HUD system provides head-up 
guidance to the pilot so, by manual control, an equivalent level of performance 
(and safety) to an auto-land system is demonstrated. A Decision Height not less 
than 50 ft. height above Touchdown Zone Elevation (TDZE) is used. 

4.3. Visual References 

The concept of visual flight references creates an important distinction for automatic and 
manual landing systems, as well as EFVS operations. For Cat. I and Cat. II approaches, 
visual reference means being able to see to land (i.e., being able to conduct a hand-flown 
landing). For Cat III approaches, visual reference means being able to see to verify 
aircraft position, if ever so briefly. It’s important to understand the use of visual 
references to appropriately gauge pilot workload and attention demands during low 
visibility approach and landing operations. 

At DA(H) or prior to the MDA on an instrument approach in other than Cat. Ill 
conditions, the pilot makes a decision whether to continue descending below DA(H) or 
MDA. This decision is made using natural vision or using an EFVS if equipped. The 
decision and the subsequent “visual segment” is based on this visual information, the 
comprehension of its meaning, and the projection of its status in the near future. 

Regulations in effect as 14 CFR §91.175 and its companion regulations under §121 and 
§135 identify the required visual references which, from a historical basis, support the 
flight crew’s informational needs to safely complete this operation. In Table II, the 
required visual landing references for §91.175 (c) (Non-EFVS flight operations) and (1) 
(EFVS flight operations) are detailed to descend below the published DA(H) or MDA 
and to descend below the 100 ft Height Above TDZE (HAT). 

From an equivalent performance perspective, the required visual references in Cat I/II 
conditions must provide information sufficient to: 

- Portray the present descent rate, its status with respect to normal operations and 

normal maneuvering limits and provide trend information that the descent rate 
and aircraft position will continually allow a safe descent to landing. 

- Provide status and trend information that may indicate if, with normal maneuvers, 

a touchdown can occur with acceptable descent rate and within the intended 
touchdown zone. 
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Ideally, during this visual flight segment, the pilot would be able to visually identify that 
the path toward the intended touchdown point is free of all obstacles, charted or 
otherwise. This task is typically performed today by operational procedure. By Terminal 
Instrument Procedures (TERPS) design, obstacle protections are provided to the landing 
runway in the visual segment of a standard instrument approach procedure and Air 
Traffic Control (ATC) procedures are in effect to keep vehicles and other aircraft clear of 
the landing runway. However, there is no instrumentation or display currently in use that 
annunciates or informs the pilot/crew that they are in the ‘approved’ visual segment 
although the use of published vertical and lateral guidance may provide this type of 
information. 

A missed approach/go-around is executed at the DA(H) or the charted missed approach 
point or so as to not descend below the MDA if the required visual references are not 
visible and distinctly identifiable or if the aircraft is not in a position to land safely. 

The task demands for the flight crew/pilot - i.e., to pick up the required visual references, 
and successfully complete their flying duties - greatly influences their time, workload, 
and ability to be aware of and detect traffic or obstacles. These task demands and by 
inference, the display designs, are affected by the assigned roles and responsibilities for 
the flight crew. 
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Table II - Required Visual References, 14 CFR 91.175 (c) and (1) 


Required Visual References Using 
Natural Vision 
(14 CFR 91.175 (c)) 

Required Visual References Using an 
Enhanced Flight Vision System 
(14 CFR 91.175 (1)) 

For operation below DA/DH or MDA - 

At least one of the following visual references for the 
intended runway must be distinctly visible and 
identifiable: 

Approach light system 
Threshold 
Threshold markings 
Threshold lights 
Runway end identifier lights 
Visual approach slope indicator 
Touchdown zone 
Touchdown zone markings 
Touchdown zone lights 
Runway 

Runway markings 
Runway lights 

For operation below DA/DH or MDA - 

The following visual references for the intended runway 
must be distinctly visible and identifiable: 

Approach light system 

OR 

Visual references in BOTH paragraphs 91.175(l)(3)(ii)(A) 
and (B) - 

(l)(3)(ii)(A) The runway threshold, identified by at 
least one of the following - 

— beginning of the runway landing surface, 

— threshold lights, or 

— runway end identifier lights 
AND 

(l)(3)(ii)(B) The touchdown zone, identified by at 
least one of the following - 

— runway touchdown zone landing surface, 

— touchdown zone lights, 

— touchdown zone markings, or 

— runway lights. 


Descent below 100 feet height above TDZE - 

At least one of the following visual references for the 
intended runway must be distinctly visible and 
identifiable: 

Approach light system, as long as the red terminating 
bars or red side row bars are also distinctly visible 
and identifiable 
Threshold 
Threshold markings 
Threshold lights 
Runway end identifier lights 
Visual approach slope indicator 
Touchdown zone 
Touchdown zone markings 
Touchdown zone lights 
Runway 

Runway markings 
Runway lights 

Descent below 100 feet height above TDZE - 

The following visual references for the intended runway 
must be distinctly visible and identifiable: 

The lights or markings of the threshold 

OR 

The lights or markings of the touchdown zone 


4.4. Crewed Procedures 

The roles and responsibilities for the pilot/flight crew are identified by the Standard 
Operating Procedures (SOPs) in effect. SOPs are universally recognized to promote safe 
flight operations (see AC 120-71 A, “Standard Operating Procedures for Flight Deck 
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Crewmembers” 27 Feb 2003). This AC provides general guidance for the development 
of SOPs for crew sharing of responsibilities and duties in low visibility approach and 
landing operations. Further guidance is available for taxi operations (AC120-74A, “Parts 
91, 121, 125, and 135 Flight Crew Procedures During Taxi Operations,” 26 Sept 2006 
and AC91-73A, “Part 91 and Part 135 Single Pilot Procedures During Taxi Operations”). 
These SOPs generally define who is responsible for the specific “aviate, communicate, 
and navigate” functions and how they are performed. These SOPs may change based on 
company/air carrier, but more typically, the SOPs change based on cockpit display 
configuration. The ACs do not, however, consider the impact of SEYS technologies and 
how roles and responsibilities may be modified by the introduction of these technologies. 

4.4.1. Pilot-Monitored Approach 

The pilot-monitored approach (PMA) is commonly employed for head-down low- 
visibility approaches. 

As noted in the Commercial And Business Aviation Advisory Circular No. 0239 (dated 8 
Sept 2006) “proper use of the PMA permits the left seat pilot to improve safety related to 
making the decision to transition from instrument conditions to visual conditions for the 
landing. During a PMA, the left seat pilot has significantly more ‘heads-up’ time for 
visual scanning outside the flight deck. This extra time permits the pilot conducting the 
landing to determine whether sufficient visual references exist to judge the position and 
rate of change of position of the airplane in order to decide to continue the approach 
visually to a safe landing.” 

This operating policy minimizes the problem of visually transitioning from instrument 
flying to visual flying. Unfortunately, this operating policy involves a change of control. 

The left seat pilot (PNF or pilot-monitoring, PM) handles communications with the 
ANSP and makes decisions, while monitoring the flying performance. The right seat pilot 
(PF) may fly the approach with or without the engagement of autopilot or auto-thrust. 

The PF is engaged directly in either: 

• Hand-flying the airplane, by flying the raw data (Cat. I) 

• Actively following the flight director commands and monitoring the raw data 
(Cat. I or Cat. II); or, 

• Supervising autopilot operation and being ready to take manual control of the 
aircraft, if required. 

The PNF scans alternatively inside and outside, calls flight parameter deviations (i.e., 
excess deviations in airspeed, pitch, bank, track, glideslope, altitude and vertical speed) 
and altitudes (e.g., “one hundred above” then “minimums”). 

Before or upon reaching the DA(H) or MDA, 

• If visual references are acquired, the left-seat pilot calls “landing” takes over the 
controls and lands; or, 
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• If visual references are not acquired, the left-seat pilot calls “go-around” and the 
right seat pilot initiates the go-around and flies the missed approach. 

If the left seat pilot commands a go-around, the right seat pilot (PF) flies the go-around 
procedure and missed approach as briefed. 

If the left seat pilot takes control for landing (now the PF), the right seat pilot (now the 
PNF) continues to monitor the aircraft performance on instruments (i.e., head-down) and 
calls out deviations. 

In this procedure, the PNF recognizes and identifies the required landing visual 
references, assesses the suitability of the landing zone, and then is tasked with flying the 
approach, landing and roll-out without the assistance of the (formerly) PF (i.e., they are 
asked to remain head-down in the event of a go-around). The PNF does not have a visual 
transition from head-down to head-up at the instantaneous moment control is transferred. 

4.4.2. Pilot-In-Active Control Loop 

The so-called “Pilot-in- Active Control Loop” assumes that there is only one HUD and it 
is equipped on the left-hand pilot side. The left-hand seat pilot flies the entire approach 
(as PF) with or without the engagement of autopilot or auto-thrust. 

The right-seat pilot (PNF or PM) handles communications with the ANSP, monitors the 
approach, and assists in the acquisition of outside visual references. The PNF scans 
alternatively inside and outside, calls flight parameter deviations and altitudes. 

The PF confirms the acquisition of visual references and calls “landing” (or “go around” 
if visual references are not adequate). 

If “landing” is called, the PF progressively transitions from instrument flight conditions 
to external visual references using the HUD. 

There is no transfer of control and the PF is head-out through the entire approach and 
landing obviating the need to visually accommodate or re-focus. The PNF is both head- 
out and head-down monitoring all systems and functions. With this procedure, two pilots 
are involved in identifying the required landing visual references and assessing the 
suitability of the landing zone. 

4.5. EFVS Procedures 

4.5.1. EFVS Crew Procedures 


Adaptations of the Pilot-In- Active Control Loop procedure have been created for EFVS 
operations. Since the regulation permits descent below the published DA/DH or MDA if 
the pilot distinctly identifies the required visual references (see Table II), additional call- 
outs and procedures are used to distinguish between EFVS required visibility and natural 
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vision and the fact that only the PF, using the EFYS, can positively identify and use the 
required EFVS visual references. 

In a crewed experiment, various display formats for the PF HUD and the PNF repeater 
display of EV imagery were experimentally tested while using these crew procedures (see 
Bailey, Kramer, Prinzel, and Wilz, 2009). In Table III, the crew call-outs are shown, 
indicating that an aditional step - the call-out and identification of “EV Lights” - was 
required to descend below the published DH. If the “EV Lights” was not called out, a 
go-around was executed at the DH. If “EVS Lights” was called, the next decision 
altitude was at 100 ft HAT where the lights or markings of the threshold or the lights or 
markings of the touchdown zone had to be distinctly identified by the PF to continue the 
approach to landing. 

The PF was responsible for flying the aircraft, confirming the acquisition of visual 
references; first, the EFVS visual references and then, the natural vision references (or 
“go around” if visual references are not adequate). The PF had the option to declutter the 
HUD and not use EV imagery for the landing; although the PFs thought the procedure to 
identify the landing references with natural vision was awkward (e.g., they would 
momentarily declutter the HUD to remove the EV imagery or look around the HUD), the 
PFs kept the EV on the HUD and used this information to complete the landing and roll- 
out (Kramer, Bailey, and Prinzel, 2009b). 

The experiment generally showed the effectiveness of EFVS and the EFVS crew 
procedures. Improvements to the procedures and call-outs were suggested, but in 
general, the roles and responsibilities of the crew using EFVS procedures extended from 
current Pilot-in-the- Active Control Loop were validated. 

Head-position and orientation tracking data was collected of the PM to quantify display 
usage and visual gaze/attention. The data showed that attention was divided between a 
repeater display of EV information, the normal flight instruments, and the forward, OTW 
view from 1000 ft AFL until 200 ft AFL. The PM scanned alternatively inside and 
outside, called flight parameter deviations and altitudes. Below 200 ft AFL, the PM was 
almost exclusively head-up, looking OTW, searching and clearing the landing runway 
area (Bailey, Kramer, and Prinzel, 2007). 
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Table III - EFVS Crew Procedures 


Altitude-Based Events 

PF 

Tasks / Callouts 

PNF 

Tasks / Callouts 

500 feet HAT 

Response: “ Systems Normal, EV 
Normal ” 

Call “500 feet” 

100 feet Above 
Minimums 

Response: “Check” 

Call “100 feet Above” 

Published Minimums 
(200 ft HAT) 

With EV Visual Cues. 
Call “EV Lights” 

When Visual Cues Appear, 
Call “Lights” or “Field in Sight ” 

Without EV Visual Cues, 
Call “Going Around” 

Without PF Call of ‘EVS Lights’. Call 
“Go Around ’ 

EFVS Decision Altitude 
(100 ft HAT) 

When Actual Visual Cues, 
Call “ Landing ” 

When Visual Cues Appear, 
Call “Lights” or “Field in Sight ” 

Without Actual Visual Cues, 
Call “Going Around' 

Without PF Call of ‘Landing’, 
Call “Go Around ’ 


4.5.2. EV Head-Down Crew Procedures 


Head-down EFYS operations are not currently authorized by the FAA; however, since 
HDDs are cheaper and more widely available than HUDs, evaluation of their potential for 
EFVS has been explored. 

Two variations of the PMA crew procedures have been tested. In both cases, the PF was 
using a HDD of EV information and guidance to descend to the published DA/DH. Two 
different piloting procedures were tested once the required landing visual references were 
identified: 

1) The PF transitions from EV head-down to head-up, using natural vision; or, 

2) Aircraft control is transferred from the head-down PF to the head-out pilot, who 
continues the approach to landing using natural vision. 

Four subject pilots flew these two HDD EFVS procedures and in comparison, a standard 
head-down PMA to 200 ft Instrument Landing System (ILS) minimums and an EFYS 
Pilot-in-Active Control Loop procedure using a HUD. 

This work showed that there were no performance differences between the procedures. 

No problems were reported in immediately identifying the runway OTW after 
transitioning from head-down to head-up. The depiction of the runway in the EFVS 
guided them to the right location to look for the runway. The post-flight debriefings 
indicated that the pilots had “good and comprehensive transition from EV head-down 
segment to visual flight without impairing flight path.” 

However, the results for the PMA procedures using a control transition at the EV 
transition height evoked some controversy among the pilots. Two pilots liked this 
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switching from PNF to PF and vice versa whereas the other two stated that this was not a 
good idea below the DA(H). One pilot had a “crash” where the landing sink rate 
exceeded acceptable structural limits. 

4.5.3. EV Single-Pilot Procedures 

Single pilot EFVS operations are not currently approved or authorized by the FAA. The 
pilot in this operation would have to successfully fulfill all required duties and complete 
all tasks currently conducted by two pilots, with acceptable pilot workload and safety 
margins. 
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5. Traffic/Obstacle Detection Using SEVS Technologies 

Within the context of the crewed and single pilot procedures for low visibility approach 
and landing operations, the influence of SEVS technologies are discussed as they pertain 
to pilots performing the role of PM or of PF. 


Visual target detection using SEVS technologies can be achieved through any of four 
possible avenues: 

1 . Using EV on a HDD 

2. Using EV on a HUD 

3. Using the OTW view, through a HUD 

4. Using the OTW view 

5.1. Automatic Target Detection and Recognition 

This work does not currently consider concepts for fused or combined EV and SV 
systems although they have been studied and some of the results are germane to this 
discussion. 

Several studies (Parrish et al, 2003; McKay et al, 2002) have shown that the optimal 
combination of SV and EV technology is one that provides the direct display of SV to the 
flight crew without direct display of EV. Instead, EV technology is used “behind the 
glass” for automatic navigation error detection, database integrity monitoring, and real- 
time obstacle/object detection. Image processing performs these functions automatically 
without intervention by the flight crew. 

This arrangement provides a highly usable display presentation (i.e., SV) that is 
unaffected by the actual weather and visibility conditions, yet if un-charted obstacles, 
traffic, database errors or navigation errors are detected by the EV running in the 
background, the situation is annunciated, and almost “perfect” decision-making by the 
pilot occurs. This advantage was also shown by Jennings and Powell (2004) where 
runway incursion scenarios were detected during simulated instrument conditions much 
more quickly using Automatic Dependent Surveillance-Broadcast (ADS-B) transmission 
of traffic data and graphical representation on an SV display than OTW visual detection. 

While degrees of success in developing these decision aids have been met, technology for 
“perfect” object detection and database/navigation error detection does not yet exist. 
Further, there may always be gaps, such as minimal radar cross-section objects or 
‘below-detection’ threshold errors, which may still warrant flight deck procedures and 
human interventions for integrity and error monitoring. Research should be conducted to 
determine the desired/required missed detection and false alarm rates for a SV which 
utilizes ‘behind the glass’ processing for automatic database, obstacle/object, and 
navigation error and omission checking (R-4). 


25 


5.2. Resolution Influence of Detection and Recognition 

In the absence of “behind the glass processing,” EV imagery has been touted as providing 
an element of traffic detection above and beyond unaided natural vision. While this 
seems intuitive, the capabilities of the imaging sensor and the prevailing weather, target 
and background determine the level of success. Detection and recognition is dependent 
upon the sensor resolution and its sensitivity to the environment and “target,” the size and 
contrast of the obstacle/object against the image background, and the display resolution 
(Mendez, Freitag, and Zinn, 1972; Arthur et al, 2005) 

To illustrate technological capabilities and limitations, Johnson’s criteria is used. The 
hypothesized scenario is a Cessna- 172 holding short for take-off the runway threshold. 
Computer-generated images of the scenario are shown in Figure 5 at 500 ft height above 
the TDZE (HAT), Figure 6 at 300 ft HAT, Figure 7 at 200 ft HAT, and finally, Figure 8 
at 100 ft HAT. The C-172 is 27 ft in length and 36 ft in span. The images show 43 
degrees horizontal field-of-view with a 4:3 H:V aspect ratio. A ‘best-case’ scenario is 
considered in these examples, where the larger dimension (span) is used to predict the 
detection and recognition range. Also, no degradation in target contrast is implied due to 
atmospheric effects or background characteristics. 

Using Johnson’s criteria (see Table I) and computing the number of pixels required as 
being twice the number of line pairs plus one, 3 and 9 pixels are required for detection 
and recognition, respectively, of the C-172. Using these assumptions, the number of 
pixels that the C-172 subtends in an image is computed in Table IV. 

Two different resolutions are shown comparing a typical EV system performance and a 
hypothetical sensor system could theoretically provide eye-limiting resolution. Present- 
day commercial FLIRs (for EFVS applications) employ approximately 320 x 256 pixel 
resolution, spanning a typical HUD Field-of-View (FOY) of 32° H x 24° Y for the 
conformal image. This provides approximately 10 pixels per degree visual resolution. In 
comparison, the Snellen visual acuity equivalent to 20/20 of 60 pixels per degree is also 
shown. The sensor in this latter case would employ 1920 x 1440 pixels. 

Table IV- Pixels Subtending the Lateral Extent (36 ft) of C-172 


Altitude 

320x256 pixel EV 
(10 pixels per degree) 

20:20 Snellen Equivalent EV 
(60 pixels per degree) 

500 ft HAT 

2 pixels 

14 pixels 

300 ft HAT | 

4 pixels 

26 pixels 

200 ft HAT 

7 pixels 

43 pixels 

100 ft HAT 

22 pixels 

136 pixels 
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Figure 5: View of C-172 Holding Short From Aircraft at 500 ft HAT 



Figure 6: View of C-172 Holding Short From Aircraft at 300 ft HAT 
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Figure 7: View of C-172 Holding Short From Aircraft at 200 ft HAT 



Figure 8: View of C-172 Holding Short From Aircraft at 100 ft HAT 
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These numbers highlight that a typical EV resolution is relatively poor and the range for 
target detection is typically much closer than if the human were viewing OTW. The C- 
172 will only subtend sufficient angular size in the EV for detection at approximately 300 
ft HAT and for recognition only after descending below 200 ft HAT. The performance, 
based on resolution comparisons, is 6 times worse than a human observer looking out the 
window. 

This is not to say that the EV can’t provide detection of the target. The advantage of an 
EV system is that it may provide a higher contrast image than the out-the-window view 
(e.g., a hot C-172 imaged against a cold runway background). It may also be that the EV 
can penetrate the atmosphere whereas the human’s natural vision is obscured. 

However, this simple example highlights that resolution is just one ingredient - but an 
important prerequisite - to traffic/obstacle detection. Performance data for target 
detection tasks must colored by the resolving capabilities of the EV and the OTW visual 
system especially when flight simulation is used. The resolution capabilities of 
simulators are typically much, much less than that of the human vision system. 

5.3. Traffic/Obstacle Detection by PM With SEVS Technologies 

In a crewed experiment, the presence and absence of symbology on a head-down repeater 
display of EV imagery for the PNF was experimentally tested while EFVS approaches 
were flown (see Bailey, Kramer, Prinzel, and Wilz, 2009). Runway incursions were 
introduced on two occasions unbeknowst to the crew and interspersed in a large number 
of nominal data runs. 

As noted by Kramer et al (2009b), almost all obstacle detections were made by the PNF 
using their natural OTW vision. 

It was hypothesized that the PNF made almost all of the detections because the PF is 
primarily focused on flying the aircraft, while the PNF, serving as a PM, had the primary 
responsible responsibility of monitoring and verifying flight path performance, cross- 
checking guidance and raw data indicators, and identifying the visual runway 
environment and clearing the landing area. 

This work also suggests that the PNF could not effectively use the repeater display of EV 
information for “clearing” the runway for obstacles. Johnson’s criteria, applied to the EV 
system and the runway “targets,” indicated that the EV information was not of sufficient 
resolution to be useful at 400 to 500 ft HAT for target detection, in the area where the 
PNF was using the repeater display (Bailey, Kramer, and Prinzel, 2007). Below 200 ft 
HAT, the EV repeater would be effective at showing traffic on the runway, but by this 
altitude, the PNF had transitioned to OTW and was not using the repeater display. 

This work showed that the PM has the attentional resources and innate responsibilities for 
runway incursion detection (i.e., target detection). However, the HDD of EV imagery 
was not tailored for this obstacle/object detection task. To detect obstacles using this 
device required higher sensor resolution to detect objects at the altitudes and ranges 
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where the PM desired to use the display (above 200 ft HAT). A zoom (magnification) 
capability would be needed to effectively increase the resolution of the sensor/display 
image before the aircraft transitioned into the visual segment. Magnification was also 
shown to be desirable in a study by Theunissen (2004). 

This work had some limitations which influence the global applicability of the data: 

1) This work did not look at the presence or absence of a repeater display. In the 
US, a repeater display of EV information is not required during EFVS 
applications; however, in Europe, a repeater display is required. 

2) This work did not look at variations in size or display minification/magnification 
effects for the repeater display of EV information. 

3) CDTI influences were not evaluated. 

4) Variations in the repeater display location were not evaluated. The PNF display 
was located outside of the primary field-of-view of the crew (see AC25-1 1, 
“Electronic Flight Deck Displays,” 27 June 2007) to the outboard and right of the 
forward instrument panel. A forward primary location may have enabled the PNF 
to quickly glance at the repeater display at lower altitudes and still maintain 
vigilance OTW. 

An evaluation of EV/SV information for the PM to perform navigation integrity, obstacle 
detection, and runway incursion detection resulted in relatively mediocre performance 
(McKay, Guirguis, Zhang, and Newman, 2002). The evaluations were flown as a single 
pilot, with the lone pilot serving as a PM. During low-visibility auto-flight landings 
where all runs included an “anomaly” (e.g., SV database misalignment, runway 
incursions, or uncharted obstacles), the evaluations showed that (see Figure 9): 

• The best performance - fewest missed detections - were found when the PM had 

automatic, ‘behind-the-glass’ detection capability (i.e., “Threat Icons”). 

• Using an EV-only HUD (“EVS HUD” in Figure 9), 57% of the obstacles were 

undetected and 50% of the runs with the database misalignment went 
undetected 38% of the runway incursions went undetected. Despite this 
relatively poor performance, the pilots subjectively preferred this 
implementation overall. 

• A Primary Flight Display concept, showing SV with a pilot-controllable EV inset 

image (“EVS Insert” in Figure 9), exhibited the worst performance for 
anomaly detection. The PFD inset concept “missed” 100% of the uncharted 
obstacles and a 33% rate of missed detections for the database misalignments. 
Detection of runway incursions showed a missed detection rate of 33%. The 
poor performance and low-acceptance of the head-down EV insert concept 
was attributed to the clutter of the image, the small EV image size, and 
confusion between the SV/EV images. A preference for control of the EV 
sensor image in an inset display has been shown elsewhere (Theunissen et al, 
2004), but pilot workload was shown to increase because of the additional 
pilot control for display management. 
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• With EV-only shown on a HDD (“EVS HDD” in Figure 9), all uncharted 

obstacles were successfully detected but 33% of the runs missed the database 
misalignment. A missed detection rate of 22% for runway incursions was 
found. This concept was well liked for its large image size and minimal 
display clutter, but disliked because of the workload in transitioning from 
head-down to head-up flight. 

The data suggests that a head-down display without imagery provides the best 
opportunity for obstacle and traffic detection. The larger the display the better, but a 
HUD presentation was preferable, especially if all symbology could be decluttered and if 
the full dynamic range of the EY image could be presented. Research should be examine 
head-down vs. head-up performance for the PM, with experimental variations in display 
size, location, and minification/magnification (R-5) 



Figure 9: PM Evaluations Using SEVS Technology 
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5.4. Traffic/Obstacle Detection by PF With SEVS Technologies 


The use of a HUD is deemed an essential “characteristic and feature” of the EFVS 
operation (FAA, 2004a) and as such, HDD implementations for EFVS are currently not 
allowed. The “essential features” of the HUD were described as follows: 

• The display should provide the EV image and spatially-referenced flight 

symbology so that they are aligned with and scaled to the external view (i.e., 
conformally drawn). 

• The display should be located so the pilot is looking forward along the flight path 

(i.e., looking at and through the imagery to the out-of-the window view) to 
readily enable a transition from EFVS imagery to the out- the window view. 

• The display should not require the pilot to scan up and down between a head 

down display of the image and the out-the- window view looking for primary 
flight reference information. This transition would be hindered by repeatedly 
re-focusing from one view to the other. 

In contrast to this mandate, the feasibility of an EFVS head-down procedure has been 
examined in simulation (Korn, Lenz, and Biella, 2007). This work was a follow-on to 
previous experimentation on crew coordination which focused principally on head-down 
displays and the application of a millimeter- wave radar sensor for EFVS (and its 
idiosyncrasies associated with transformation from an azimuth and range data to an 
egocentric, perspective display) (Lorenz and Korn, 2004). This work suggested that 
head-down operations in low visibility conditions was feasible, although not necessarily 
without potential limitations, especially if a change in aircraft control, from PF to PM, 
was necessary at low altitude. 

Head-down to head-up transitions from instrument to visual flight conditions without a 
HUD - the process of looking up from a head-down instrument panel, refocusing and 
accommodating to the outside visual field, and perceiving and reacting to visual stimuli 
in the external scene - may take as long as 2 to 5 sec (Fisher, 1979). Pilots were observed 
to make from 4 to 14 separate head-down to head-up transitions during low visibility 
approaches, dependent on the simulated visibility and ceiling height. More transitions 
were made for higher ceiling heights and visibility conditions as the pilot’s maintained a 
continuing cross-check between the head-down instruments and external visual cues. 

The advantages of a HUD by minimizing visual scanning, accommodation, and re- 
focusing when transitioning from instrument to visual flight conditions are well founded 
in research (Wickens and Long, 1994; Lauber et al, 1982; Martin-Emerson and Wickens, 
1997; Weintraub and Ensing, 1992; and Newman, 1995). The HUD visual transition 
effects have even been shown in ground vehicle transportation studies (e.g., Liu and 
Wen, 2004). These perceptual advantages were felt to contribute toward superior lateral 
touchdown performance using a HUD compared to HDDs for a transition to visual 
references, although no improvement in longitudinal touchdown performance was shown 
(Goteman, Smith, and Decker, 2007). 
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Wickens and Long (1994) also reported a significant improvement in tracking 
performance prior to breakout when GA pilots flew a HUD in an ILS approach task. The 
improvement was attributed to the fact that pilots didn’t have a need to “divert attention 
away from the symbology” when head-up to find the runway at the break-out from 
instrument conditions. This observation was also supported by an (non-significant) 
improvement to recognize and confirm when the runway first became visible. 

An attempt was made to quantify how these perceptual demands might impact 
quantitative approach performance. Performance-based approach standards from the 
Joint Aviation Authorities Joint Aviation Requirement All Weather Operations were 
applied to capture the effects of transition from head-down to head-up using natural 
vision or head-up EFVS to head-up natural vision (Kramer, et al, 2009a; Kramer et ah, 
2009b). These analyses showed that eliminating any visual transition improved tracking 
performance. However, the differences were not large and not statistically significant. 
Nonetheless, the trends make intuitive sense. By allowing the pilot to concentrate on the 
task and display information, improved flight tracking performance results. The HUD- 
visual segment transition allows the pilots to simultaneously perform glideslope 
corrections and acquire the required landing visual references through normal vision. If 
the pilots did not have to shift their attention, tracking performance improved. 

These HUD advantages are not, however, provided without complications, limitations, or 
penalties as discussed in the following. When using HUDs (or other virtual displays), the 
pilot’s accommodation and cognition are influenced by numerous complex and 
interacting HUD factors, creating sometimes contradictory findings (Edgar, 2007). From 
Edgar (2007), these factors include: 

1 . The HUD image occludes parts of the background; 

2. The HUD image is displayed proximal to the use (i.e., the HUD combiner), near 
an individual’s accommodation distance; 

3. The user’s attention is directed to the HUD image; 

4. The HUD presents information to be processed by the user; 

5. The HUD image is monochromatic; 

These factors have been historically, but not necessarily accurately, lumped into two 
issues: clutter and attention capture. These are discussed in the following in reference to 
obstacle and traffic detection. 

5.5. Clutter 

Display clutter, especially for HUDs, has historically been a concern (e.g., Newman, 
1980) and the increased use of EFVS for HUDs pushes this issue to the forefront (e.g., 
see Zuschlag, 2001). The goal of a display design should always be to minimize visual 
clutter. Only graphical elements that “add useful information content, reduce flight crew 
access or interpretation time, or decrease the probability of interpretation error” should be 
used (AC 25-11). 
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One universal definition of visual clutter is not available. Various clutter definitions have 
been proposed which typically group by (Kaber et al, 2008): 

• Content or format of the data 

• Degraded performance of the operator using the display 

• Irrelevant information to the task at hand 

The current clutter definition under AC25-1 1 (“Electronic Flight Deck Displays”) 
includes each of these properties: 

• “A cluttered display presents an excessive number or variety of symbols, colors, 
and/or other unnecessary information and, depending on the situation, may 
interfere with the flight task or operation. A cluttered display causes increased 
flight crew processing time for display interpretation, and may detract from the 
interpretation of information necessary to navigate and fly the airplane.” 

Note that the use of a “de-cluttering” function is also formalized for the express purpose 
“to enhance pilot performance” under AC25-1 1. In fact, under the EFVS rule within 
§91.175(m), the “displayed imagery and aircraft flight symbology do not adversely 
obscure the pilot’s outside view.” Under the MASPS for EFVS and SV (RTCA DO- 
315), requirements are contained such that “a control shall be provided which permits the 
pilot flying to deactivate and reactivate the display of the EFVS image on demand 
without removing the pilot’s hands from the primary flight controls (yoke or equivalent) 
and thrust control.” 

Display clutter contains two critical aspects. 

• First, clutter exists from a “bottoms-up,” data-driven aspect, addressing the effect 
of clutter to confuse or obscure visual information in terms of the “physical” 
attributes of the display due to the number of display elements, their size, form, 
proximity, and luminance (Alexander et al, 2008). These issues are influential 
both in a local domain, referring to the area immediately surrounding “critical” 
display areas and in the global domain, associated with entire display and display 
area, in general. 

• Display clutter also includes a “top-down,” knowledge-driven aspect, addressing 
the effect of visual information on human cognitive processing and attention. The 
issues of particular interest are the relevancy and redundancy of the displayed 
information. Clutter involves the display content and its influence/relevancy to 
the piloting task at hand (Kaber et al, 2008). 

Display clutter, therefore, involves the physical attributes of the display and its content 
and its influence/relevance y to the piloting task and cognitive/attentional demands 
(Kaber et al, 2008). Experimental evidence suggests that pilot experience modulates the 
perceptions of clutter with high-time, HUD experienced pilots being more accurate and 
consistent in judging the occurrence of clutter. Pilots with prior HUD experience also 
appear to be more consistent in relating display clutter to perceptions of cognitive load. 
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Clutter is linked - as discussed below - with attention capture. 

A key element that is missing in this foundational work is the influence of actual HUD 
operations in relevant weather and operational contexts especially considering the 
inherent luminance limitations of the HUD and the seemingly unlimited luminance and 
coloration of the real-world. Research should be conducted evaluating qualitative and 
quantitative metrics of clutter in actual flight conditions (R-6). 

5.6. Attention Capture 

Attention capture is the phenomenon wherein a pilot fixates their attention to one 
task/element at the expense of ignoring other potentially critical task information. This 
phenomenon has been reported in human factors and aviation psychology research 
literature and attributed to several accidents, particularly associated with the use of a 
HUD (e.g., see Crawford and Neal, 2006; Prinzel and Misser, 2004). In those cases where 
it was observed, the authors typically concluded that pilots flying with a HUD were less 
likely to detect an unexpected “event” in the “far domain” because of their fixation on the 
HUD and its symbology (i.e., the “near domain”). 

Because of the great emphasis on attention capture with HUD usage, and its potential role 
in traffic/object awareness and detection, pertinent literature are reviewed. 

The most often cited case of HUD attention capture was a simulation experiment using 8 
pilots (Fischer, Haines, and Prince, 1980) flying a low visibility approach and landing 
task. Four runs were flown without expectancy for the runway incursion with the HUD; 
four runs without HUD. The simulated weather had a 180 ft cloud deck, in 2000 ft RVR. 
With this weather, if the aircraft were on glideslope, the pilots might be able to first 
detect the incursion at approximately 1 60 ft HAT. 

The data, at first glance, makes a compelling case for HUD attention capture. In all of 
the non-HUD runs, the flight crew noted the incurring vehicle. In half of the HUD runs, 
the runs “resulted” in an incursion. 

Upon closer inspection, however, the data (shown in Figure 10) contained many 
unintentional confounds and it isn’t nearly as neat as described as above. First, two of 
non-HUD runs were spoiled by experiment protocol. The confederate FO inadvertently 
contaminated or terminated the runs. Thus, two out of four non-HUD runs did not result 
in an incursion; the other two didn’t count. Second, one of the HUD runs which was 
claimed to “result” in a runway incursion was actually terminated at an altitude higher 
than one of the “detected incursions” in the no-HUD condition (i.e., the HUD run 
terminated at 67 ft HAT, whereas the pilot in a non-HUD run spotted the incursion at 63 
ft HAT). Finally, inspection of the data showed that the non-HUD pilots had relatively 
poor ILS tracking performance and were significantly below the glideslope, resulting in a 
longer time for “visual conditions” than the HUD pilots who flew on glideslope. As a 
rule of thumb, the pilot only had ~10 seconds from 160 ft HAT to 60 ft HAT to see the 
traffic and initiate a go-around or the confederate FO would terminate the run. 
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(Note that their second runs - with full expectancy of an incursion event - resulted in 
better detection performance with the HUD. The pilot’s were flying head’s up and 
therefore, had better performance since they didn’t have to transition from head-down to 
head-up.) 

Despite these confounds, the “attention capture” conclusion from this work spawned 
numerous other research efforts to identify its physical and/or psychology mechanisms. 
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Figure 10: Data From Fischer et al (1980) For Runway Incursion Event 

Excellent summaries of attention capture and associated research are contained in Prinzel 
and Risser, (2004), Crawford and Neal (2006), and Edgar, 2007. Weintraub and Ensing 
(1992) suggest that cognitive switching is “impeded by: a) optical clutter; b) the 
conformal aspects of the symbology; c) the lack of physiological change (e.g., in 
accommodation, in convergence, in gaze angle) to serve as cognitive signals to switch 
attention.” The cognitive switch might also be impeded by the compelling nature of the 
HUD flight-director/command symbology. 

With 20/20 hindsight provided by the many years and the numerous studies conducted 
since on this subject, six factors seem to emerge as profoundly influential in HUD 
attention capture research: 

1. OTW Visual Fidelity 

2. HUD Optics 

3. Conformal Symbology 

4. Brightness, Contrast, and Declutter 
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5. Type of Event 

6. Pilot Experience and Training 

These factors follow Edgar’s list of confounding factors (Edgar, 2007) in accommodation 
and cognition using virtual displays and include some of what is already known about 
perceptual grouping and attention capture for virtual displays and HUDs (see for 
example, McCann, Foyle, and Johnson, 1993). 

More importantly, these factors are expanded upon as they are not often considered or 
factored into the empirical/applied aviation research which identified “attention capture” 
as a safety factor in the use of HUDs. These factors possibly describe how attention 
capture “problems” found in laboratories, run counter to operational data: 

• A study by the Flight Safety Foundation (2009) found that HUDs prevented or 
positively influenced 38% of accidents overall in modem glass cockpits. Of these 
accidents where the pilot was directly involved, such as takeoff or loss-of-control, 
the “safety advantage” of the HUD became much higher (69% and 57%, 
respectively). 

• Development and certification flight testing of Head-up Guidance Systems for 
manual Cat. Ill landings (Green, 1988) initially included concerns for HUD 
attention capture. However, test experience noted that “a pilot can become 
engrossed with any control task. Fixation, or a narrowing of his horizon of 
perception to extraneous cues, can occur heads-down in a tracking task as well as 
heads-up.” The flight data noted that “at no time in the flight program did any 
pilots comment about a fixation problem associated with the HUD, or notice any 
reduced capability to either scan for traffic or detect unexpected traffic while 
maneuvering in a terminal area.” 

5.6.1 OTW Visual Fidelity 

A critical aspect of attention capture is the saliency or similarity between the HUD (“near 
field”) and OTW visual field (“far field”). As noted by Crawford and Neal (2006), “if the 
HUD is more salient of the two domains, these elements of the external world, especially 
those that are unexpected, may be more difficult to detect.” 

The OTW visual fidelity in laboratory or simulator evaluations creates a critical 
difference of HUD attention capture compared to the flight environment. Notable 
laboratory / simulation tests showing attention capture (e.g., Hofer et al, 2000; Fischer, 
Haines, and Prince; 1980; Wickens and Long, 1995) provided 20:60 visual acuity at best. 
Based on acuity alone, with these degraded visuals, pilots must be twice to three-times as 
close to an incurring vehicle for detection as could be expected in-flight. As noted in 
Section 5.2, the visual acuity of present-day EFVS HUDs provides only 10 pixels per 
degree resolution - one-sixth of a human’s normal visual acuity. Therefore, in-flight, 
there is a significant difference in acuity between the HUD and the OTW, promoting 
salient differences. In laboratory or simulator evaluations, these acuity and saliency 
differences may not be present. 
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Because of degraded OTW visuals, pilots were deprived of or received degraded OTW 
visual cues critical to the task. In the absence of OTW visual cues, pilots can become 
reliant on the HUD guidance symbology for flight control, drawing attention and focus 
unnaturally toward the HUD at the expense of the OTW (Weintraub and Ensing, 1993). 

Another notable component in far-field/near-field saliency is the visual cueing/fidelity of 
the OTW target depiction and lighting. Especially in low-visibility and night operations, 
airport and traffic lighting is a critical visual cue. Calligraphic lighting is critical 
simulation tool to mimic realistic lighting effects; these point light source effects were 
notably absent in almost all instances of HUD attention capture research. Their absence 
degrades the saliency of the OTW scene and thus, promotes HUD attention capture. One 
could argue that not all “targets” of interest have lights. However, even though a target 
may not have lights, the fact that a repeatable pattern of lights is absent due to 
obscuration or shadowing also provides visual cues (e.g., see AC91-73A). 

Emerging technology trends suggest possible trending toward HUD attention capture: 

• State-of-the-art Liquid Crystal Display HUDs are moving to much higher 
resolutions, nearing eye-limiting acuity. This trend would suggest a reduction in 
saliency differences between the HUD and OTW visual fidelity. 

• Fortunately, the underlying resolution of FLIRs are remaining, for the foreseeable 
future, at approximately 320 Horizontal x 240 Vertical pixel resolution because 
higher FLIR resolution would trigger unwanted export restrictions associated with 
International Trafficking Acquisition Regulations. (The lower resolution FLIR 
signals are being mapped to the higher resolution HUD video formats.) 

• EFVS HUD operations would suggest a trend toward HUD attention capture 
given that, by definition, the HUD imagery contains more information than 
provided in the natural vision spectrum. The HUD attention capture literature is 
also devoid of this effect, as discussed in Section 5.3. 

Research should be conducted using high fidelity simulation and flight testing; evaluating 
HUD attention capture effects in very low visibility operations (visibilities less than 700 
ft) where the saliency becomes less distinct (R-7). 

5.6.2 HUD Optics 

A vast majority of HUD attention capture research didn’t, in fact, use a real HUD, but 
instead, simulated HUD operations by embedding the HUD imagery into the OTW visual 
scene (e.g., Fischer, Haines, and Prince, 1980; Weintraub, Haines, and Randle; 1985; 
Wickens and Long, 1995; Ververs and Wickens, 1998; Hofer, et al, 2000). The 
simulations created the case where the OTW and HUD imagery were at the same optical 
distance. Using this simulation methodology, unfortunately, masks the perceptual 
differences of the HUD and the OTW. It also emphasizes obscuration of the OTW by 
HUD symbology. 
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HUDs also include other critical factors which optically promote the differentiation 
between the HUD and the OTW visual scene: 

1. Real HUDs are not focused at infinity (Gibson, 1980). 

The design goal of a HUD is to provide a virtual image at optical infinity but 
HUD optical systems are not perfect; there are residual uncompensated optical 
errors in the optical system. As such, HUDs are typically focused inside of 
optical infinity. (As described in SAE AS8055, minimum acceptable errors in 
monocular positional accuracy are specified. The complication arises that these 
errors can cause binocular parallax errors. Stringent requirements are in place to 
limit optical errors which might cause a pilot’s eyes to angle outward (divergent 
errors) because pilots cannot easily accommodate these errors (Gibson, 1980). As 
such, HUD optics are typically biased toward convergent errors (apparent display 
depth to vary inside of optical infinity across the field-of-view) but beyond 
resting-eye focus. The net effect is that perfect conformal overlay is not provided 
by a real HUD. Head movement will generate some degree of differential motion 
between the HUD symbology and the OTW visuals, thus, promoting attention 
switching.) 

2. Pilots don’t have perfect vision: 

Disparity between the HUD virtual image and the OTW imagery is further 
differentiated by pilots not having perfect vision. Any disparity in visual 
performance between a pilot’s left and right eye will exacerbate the imperfection 
from an ideal “optical infinity” design. 

3. HUDs have distinct eye boxes: 

HUD optics have a distinct eye box over which the exacting design requirements 
for monocular and biocular performance are specified. However, outside of this 
eye box, the pilot can view an unobstructed, uncluttered view of the world by 
head movement, which clearly separates the “near domain” and “far domain” 
information. 

4. Real HUDs have limited luminance and distinct coloration: 

P-43 phosphor provides a unique green color which is almost unique in the 
outside world, so the probability of “color obscuration or clutter” with an object in 
the real-world is improbable, lending support that the HUD will be perceived as a 
separate entity from the OTW, supporting attention switching (McCann, Foyle, 
and Johnson, 1993). 

Emerging technology trends are, again, trending toward HUD attention capture 
principles: 

• Color HUDs are not yet a present-day reality but if they come to fruition, very 
careful color coding will be necessary to ensure a minimum of color-obscuration 
and to support attention/domain switching by color differentiation. 

• Optics designs are emerging to create much larger design eye boxes. Increasing 
the size of the eye box would lessen the perceptual switch between the “near 
domain” HUD and “far domain” OTW visual information. 
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Research should be conducted evaluating the impact of color HUDs (and head-worn 
display or virtual display) on attention capture, clutter, and target detection (R-8). 

5.6.3 Conformal Symbology 

Human information processing is constrained by attention limitations. Scene-linked 
symbology has been shown to be a necessary but perhaps not sufficient means to support 
simultaneous processing on instrument information and far domain information. 

(McCann and Foyle, 1995). This use of “scene-linked symbology” - where these 
symbolic references are located such that they overlay a real-world position and move 
and transform as though they were actual objects in the world - facilitates efficient 
cognitive processing and mitigates problems of attention tunneling and symbology 
fixation. 

Real HUDs are composed of conformal and non-conformal elements. This mix of 
symbology types emphasizes attention/domain switching via differential motion 
(McCann and Foyle, 1995). 

HUD symbolic changes and aural cues are employed by design and/or standard operating 
procedures to ensure attention/domain switching. For instance, HUD systems may use 
conformal runway edges for visual momentum and acquisition on the approach, but they 
disappear at 50 ft, creating a visual change - a visual vacuum as it were - that is filled by 
the outside runway edge environment (Green, 1985). Automatic or crew call-outs of 
altitudes and runway status aid this attention switch. 

Perfectly conformal symbology in real HUDs is not possible. In Section 5.2.2, the optical 
distortions of real HUDs were discussed and these negatively impact conformality. 
Further, the latency involved in signal processing to sense, compute, and draw conformal 
HUD imagery creates inaccuracies in its representation against a real-world object. 
Excessive delay creates symbology “swimming” which, as the latency increases, 
eventually becomes unacceptable (Bailey et al, 2004). 

Research should be conducted evaluating the impact of imprecision in conformal HUD 
symbology (statically and dynamically) on attention capture effects and the ability of the 
pilot to use EFVS imagery especially as it is the predominant source of visual 
information for landing in very low visibilities (<700 ft visibility) (R-9). 


5.6.4 Brightness, contrast, and declutter 

HUD attention capture (and clutter) has been shown to be dependent upon the symbology 
contrast (Ververs and Wickens, 1996 and 1998). This result is intuitive, given that the 
apparent contrast (luminance of the “near domain” symbology against the background 
“far domain” background) directly impacts saliency. 

Unfortunately, this degree of freedom has too often been an experimental confound in the 
attention capture research. It was noted in Hofer et al (2000) that “one of the pilots 
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commented that the HUD was worse for switching and would induce tunneling, when the 
tactical symbology was too bright; this is why it was suggested to turn down the 
symbology brightness to make it less compelling and to detract less of the pilot’s 
attention.” Ground simulation is notoriously poor at reproducing real-world HUD 
operations - critical in the evaluation of cognitive switching (and the potential for 
cognitive capture) and occlusion. 

While real-HUDs may be used in simulation - simulating eye box effects and imperfect 
optics - the evaluations are compromised by the very limited luminance output of the 
external scene, out-the-window projectors. The limited luminance (less than 50 ft-L 
typically) of the OTW often creates a biased evaluation between the HUD imagery and 
the OTW - critical for object and obstacle detection - especially without experimental 
control of the HUD brightness settings. The transmissivity of the HUD combiner also 
corrupts the contrast and brightness of objects when looking through the HUD to the 
point where they may not be perceptible. “Burn-through” of landing lights is an 
important visual element of HUD low visibility EFVS operations which is not often 
simulated because of the limited luminance output of the OTW visuals and the HUD 
combiner transmissivity. Thus, the low luminance of the OTW background may bias the 
perception of the HUD imagery as it depends upon the background illumination. 

Emerging technology is trending toward HUD attention capture concerns: 

• Emerging Liquid Crystal Display HUDs replacing CRT-based systems can 

provide much higher luminance outputs, enabling full sunlight-readable imagery. 
Moving toward much higher luminance levels would suggest: a) a possible 
reduction in saliency differences between the HUD and OTW visual fidelity; and 
b) a higher propensity for symbology obscuration of OTW visual information. 

Research should be conducted evaluating the impact of HUD brightness as well as 
simulated OTW visual luminance (apparent target contrast) on attention capture effects 
and the ability of the pilot to detect target (R-10). 

5.6.5 Type of Event 

A meta-analysis found that, although improved tracking performance can obtained with 
HUDs, “detection performance was found to be worse when a HUD was used during the 
landing phase of flight, a HUD cost that is amplified when events are unexpected” 
(Fadden, Wickens, and Ververs, 2000). 

Hofer et al (2000) also found that HUD performance was inferior to HDDs in expected 
event detection, and far inferior in the detection of significant events leading to a possible 
incident or accident (i.e., runway incursion). (Nevertheless, the subjective results 
revealed that the pilots judged the HUD as being lower in overall workload and more 
usable than the HDD. The perception also was that it is easier to switch between the 
displayed information on the HUD and the outside scene, than it is to switch between the 
HDD and the outside scene.) 
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As noted by Wickens and Alexander (2009), “the implications are not that HUDs are 
problematic, but rather than imposing an event or visual item within the foveal vision 
does not guarantee its detection, if it is unexpected and not salient.” 

In contrast, in terms of responding to unexpected occurrences, the detection rates and 
response times were comparatively higher and faster, respectively, when subjects used a 
HUD (Weinrauch et ah, 1989; Sojourner and Antin, 1990); however, reaction times were 
faster using the HUD only in a low-workload situation. When the workload condition 
was high, on the other hand, the HUD users had longer reaction times when compared to 
the HDD (Fischer et al., 1980; Iarish and Wickens, 1991; Wickens et al., 1993). 

Automotive HUD research (Wolffsohn, McBrien, Edgar and Stout, 1998) has shown that 
response times to and detection of changes in the HUD image and outside world scene 
were significantly worse with increased cognitive demand. Response times to and 
detection of changes in the HUD image and outside world scene also increased with age. 

Research should be conducted evaluating the impact of experience and training in HUD 
operations for runway incursion detection and attention capture effects (R-ll). 

5.6.6 Pilot Experience and Training 

Another unfortunate confound in the HUD attention capture literature is the experience 
base of the subjects. HUD experience has been found to be critical in understanding and 
correct use of HUDs (Kaber et al, 2008). 

Many of the subjects in attention capture research were General Aviation (GA) pilots, 
(e.g., Wickens and Long, 1993) without prior HUD experience and even in those tests 
that used highly experience flight crews, HUD experience was limited or non-existent 
(e.g., Hofer et al, 2000; Atkins et al, 2001). As Hofer (2000) noted, “it was commented 
that a major training effect is to be expected with the HUD. That is, it is felt that pilots 
can be trained at what they need to do and get significantly better at performing all 
required tasks at a desired level of performance. To rely solely on the HUD is considered 
a learning curve, but the learning curve is expected to be moderately steep.” 

HUD experience brings two key points: 

• Understanding of HUD brightness and implications for clutter and how to 
declutter the HUD effectively; 

• Understanding of HUD attention capture and transition to the OTW. Hofer et al, 
2000 theorized that pilots are “trained to make a physical switch between the two 
planes of information (head-up OTW and head-down) in a non-HUD aircraft;” 
whereas, using a HUD, a deliberate “attention” switch is required of the pilot to 
shift from information gathered from the HUD-symbology (near field) to the 
OTW visual (far field). As noted (Weintraub and Ensing, 1992), “when switching 
from a head-down display instrument panel to the outside world, there are three 
strong cues conveying the message that a switch is in progress from display 
information to outside scene information: The pilot looks up, the pilot changes 
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focus, and the pilot changes convergence form the outside world. They are 
powerful reminders to switch attention.” 

A pilot can become engrossed with any control task. Fixation, or a narrowing of their 
perception to other cues, can occur heads down as well as heads-up. This attention 
switch during HUD operations is promoted by judicious use of HUD brightness. For 
instance, the importance of HUD brightness and declutter controls are commonly trained 
in HUD operations (G. Saylor and R. Moreau, personnel communication, 29 May 2010) 
although standard operating procedures (SOPs) are not in place nor are SOPs expected 
since straight-forward rules for SOPs aren’t possible or practical. Training emphasizes 
the influence of HUD brightness, clutter/occlusion, and declutter controls. It stresses that 
the HUD brightness should be set as low as possible to keep the pilot looking through the 
HUD, yet still being able to see the HUD information. Pilot experience directly 
influences HUD brightness usage, with more experienced pilots tending to use less 
brightness as they have learned and become proficient in the HUD, its symbology and 
usage. 
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6. Cockpit Display of Traffic Information with SEVS 

The ability of a human to detect obstacles and traffic visually has been discussed and the 
impact of using a HUD or SEVS information on HUDs and HDDs has been discussed. 
This ability is also modulated by the roles and responsibilities defined for the flight crew 
(pilots). The emergence of ADS-B for Cockpit Display of Traffic Information (CDTI) 
creates the potential for significant improvements in traffic awareness and detection. 

Four tactical and strategic capabilities/operational benefits (McAnulty and Zingale, 2005) 
are envisioned by use of CDTI: 

• traffic awareness for the flight crew; 

• airborne spacing where the flight crews achieve and maintain a given spacing 
with a designated aircraft, but controller remains responsible for separation; 

• airborne separation where controller delegates separation relative to designed 
aircraft to the flight crew; or, 

• airborne self-separation where a controller delegates separation relative to all 
known aircraft in accordance with applicable airborne separation minima. 

Numerous references describe potential applications for employment of CDTI (e.g., see 
McAnulty and Zingale, 2005; RTCA DO-289, MASPS for Aircraft Surveillance 
Applications). This technology is clearly one of the foundational enabling technologies 
for NextGen. These works have been expanded into government and industry consensus 
leading to Minimum Operational Performance Standards and Safety, Performance, and 
Interoperability Requirements for numerous CDTI-based operational capabilities. 

All of these works are pertinent but in this report, the use of CDTI for awareness of 
surface traffic is examined. In particular, research are reviewed as to how CDTI supports 
and/or modifies the pilots’ task of traffic detection and collision avoidance in the terminal 
maneuvering area (i.e., runway and low altitude safety) and during surface operations. 

6.1. Object/Traffic Detection and Use of CDTI 

The process of “visual scanning” by a pilot (crew) is greatly influenced by the search 
importance, vigilance, and localization, such as: 

1) The current tasking of the pilot (crew) - i.e., what are the roles and 
responsibilities of the flight crew in the operation which dictates the time and 
attention available to perform a visual search. 

2) Whether the pilot’s search is localized or directed to look for a “target” in a 
certain area or location, as compared to very large, generalized area. 

3) Whether the pilot is alerted to potential collisions or conflicts with traffic or 
objects, thereby, increasing the pilot’s motivation and urgency in conducting a 
success search. 

CDTI has been shown to be an effective enhancement for visual acquisition of traffic 
(e.g., see Battiste, Ashford, and Olmos, 2000; Bone, Helleberg, and Domino, 2004; 
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Moore, 1997; Andrews, 1984 and 1991). CDTI creates an “alerted” search situation and 
also provides localization for the visual search. 

Some studies show an eight to nine times improvement in detection when alerted 
(Andrews, 1991; Boff and Lincoln, 1988). Un-alerted pilots were noted to concentrate 
their gaze in the forward direction (within 30° of straight ahead), but when informed that 
they were on a collision course, spread their glances more evenly over the visible area 
(Howell, 1957). Flight deck observations noted that, when available, CDTI was the first 
method used in the process of visually locating traffic (Joseph, Domino, Battiste, Bone, 
and Olmos, 2003), providing information for the pilots to localize their OTW visual 
search. CDTI provided increased positive visual acquisitions (Prinzo, 2003). However, 
dedicated studies in the use of CDTI (without collision detection and alerting) for traffic 
awareness and detection on the runway are few. 

For target detection using CDTI, operational differences (e.g., Part 91 versus Part 
121/135 operations) and airspace class differences likely influence CDTI usage, 
modulating the amount of head-down time caused by CDTI (Wickens et al, 2001). For 
instance, significant time is already required for head-down attention to the instruments 
for aircraft control - some aircraft and operations more than others. Airspace class 
dictates equipage and operational procedures so pilots may not have to attend to OTW 
scanning as much in controlled airspace as VFR airspace. These changes affect the 
“value” that the pilot places on OTW scanning viz-a-viz CDTI and head-down scanning. 
If the aircraft is operating in higher traffic density conditions or in uncontrolled airspace, 
OTW and CDTI vigilance changes because the “value” of scanning is significantly 
higher. 

Wickens et al (2001) found that pilots used CDTI to guide them as to where to look to 
detect traffic, providing 5 seconds faster detection of non-conflict traffic than were pilots 
in the non-CDTI condition. However, during traffic conflict trials, pilots allocated 
relatively more attention to the CDTI, in order to plan their avoidance maneuvers, and 
this led to slower OTW detection times than in the baseline. 

Head-down time is a recurring, significant issue for CDTI suggesting that the use of 
CDTI draws pilots into the cockpit at the expense of scanning OTW for traffic (e.g., see 
McAnulty and Zingale, 2005; Battiste, Ashford, and Olmos, 2000). For instance, less 
head-down time and better integration of CDTI into the operation was found (Bone et al, 
2003) when using a forward, primary FOV CDTI versus a CDTI display located near the 
center aisle-stand/throttles. Pilots may be able to effectively integrate CDTI into this 
head-down scan. Workload was also found to be lower for the forward, primary FOV 
CDTI, but these differences were not considered to be operationally significant. An 
earlier study by Abbott and Moen (1981) had essentially reached the same conclusions. 

General design principles have recently been developed for 2-D traffic display 
symbology as part of RTCA SC- 186 (Zuschlag, Chandra, Helleberg and Estes, 2010) 
with emphasize on how the symbology promotes intuitiveness, ease of learning, and ease 
of remembering. The additional of directional information to CDTI traffic symbols 
during airborne collision encounters was shown to not negatively affect pilot response to 
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TCAS resolution advisories (Olsen et al, 2009). The directional information also didn’t 
affect pilot scanning. 

Research should be conducted evaluating the use of CDTI for traffic awareness and 
detection on the runway without traffic collision detection indications or alerting. This 
work would be complementary to research on indications, warnings, and alerting 
methodologies to establish the cognitive and human-centered and engineering design 
trade-offs of Terminal Maneuvering Area Conflict, Detection, and Resolution systems (R- 
12 ). 


6.2. Object/Traffic Detection and Use of CDTI with SEVS - Exo-Centric 

The criticality of head-down time significantly influenced the development of airport 
moving map concepts (i.e., exocentric view of a synthetic airport layout, geo-referenced 
to ownship and CDTI) especially for the Taxiway Navigation And Situation Awareness 
(T-NASA) concept (Foyle, Andre, McCann, Wenzel, Begault, and Battiste, 1996). The 
T-NASA moving map was designed to promote eyes-out navigational awareness rather 
than eyes-in control functionality. 

Eye-gaze tracking and analysis indicated that the amount of time dwelling on the moving 
map was significantly different depending upon the visibility condition (Graeber and 
Andre, 1999) but the operational significance of these differences (ranging from 38% to 
41%) is questionable. Pilots who were instructed on the intended use of the moving map 
gazed at the map less (35%) as opposed to those that were not instructed (44%). Using 
audio or visual "head-up" callouts as to the pilot’s visual attention to an airport moving 
map reduced the visual attention demand of the airport moving map display, without 
significant performance degradation relative to the no-callout condition (Purcell and 
Andre, 2001). 

As NextGen operations emerge - including concepts for 4D surface operations requiring 
runway to gate time-of-arrivals - the influence of CDTI, moving map concepts, and head- 
down time warrants renewed investigation. A study of these concepts (Shelton et al, 
2010) using “non-normals” to assess safety-critical scenarios showed that pilots were 
actively using and interacting with the surface map display concept only when there was 
valuable surface traffic and intent information. When flying ownship-only display 
concepts, the map was useful to support route and navigation awareness, but it was not 
often needed. But as traffic intent information and cleared route information was added, 
additional head-down time was attracted (Prinzel et al, 2010). The non-normals results 
showed the value of surface traffic CDTI and surface traffic ‘intent’ information but 
many of these potential incidents or accidents in the non-normals were also effectively 
mitigated by an EV-equipped HUD. Another study showed very similar results where 
four non-normals, involving taxiway or runway incursions studying the effect of the 
presence or absence of ‘party-line’ information were effectively mitigated by an EV- 
equipped HUD instead (Prinzel et al, 2010). The role of the crew (pilots) and the fact 
that the pilot- flying must remain vigilant and head-out during surface operations 
significantly modulates the effectiveness of CDTI and how effective CDTI can be 
depends upon the display location and its content. 
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The integration of CDTI with SEYS technologies must consider the presentation format 
and especially, the potential for clutter as traffic and terrain/flight reference information 
must be effectively integrated and quickly interpretable. Co-planar CDTI formats suffer 
from scanning-related integration that increases with conflict density. However, 
perspective CDTI formats suffer from perceptual ambiguity, such as range-to-traffic 
estimation difficulties (Alexander, Wickens, and Merwin, 2005). The use of highlighting 
can be useful but must be appropriately tailored (Johnson, Liao, and Tse, 2003). 

A runway incursion prevention system has been created to add another “safety wrapper” 
around the pilot/crew awareness provided by the airport moving map (such as T-NASA). 
The system adds aural and visual alerting of the incurring traffic to create awareness and 
localize the visual search. The importance of an ‘alerted’ search was highlighted in a test 
of incursion prevention concepts for GA pilots. An airport moving map with traffic was 
found to be only marginally beneficial without alerting. One pilot experienced a severe 
runway incursion despite having the traffic displayed on an airport moving map (Jones 
and Prinzel, 2006). The incursion resulted because the pilots had transitioned to out-the- 
window and were not focused on the head-down display to locate incurring traffic. It was 
hypothesized that the utility of the surface map would be more effective if the display 
was located higher on the instrument panel, closer to the pilot’s head-up eye gaze. 

Continued enhancements for traffic awareness on airport moving maps are emerging, 
including initiatives under SC- 186 Surface Indications and Alerts (SURF-IA) providing 
obvious and direct indication of runway occupancy and potential safety/runway traffic 
implications (Moertl and Nickum, 2008). Further, NextGen concepts for 4D trajectory 
based operations, including required time of arrivals in airborne and surface operations, 
are spawning new CDTI such as the explicit depiction of cleared taxi routes for ownship 
and traffic with their required and actual taxi performance graphically drawn (see Figure 
11 ). 

The importance of CDTI and advisory, caution, and warning systems for NextGen 
include low altitude operations where desires for increased airport throughput will dictate 
more closely spaced arrivals. In an experimental simulator study (Pritchett and Hansman, 
1996), traffic displays with increasing amounts of traffic information were evaluated in 
closely-spaced parallel approaches. The results generally support the contention that with 
more and better trend information, improvement in performance for conflict detection can 
be attained. 

Research should be conducted evaluating the optimal methods for integration of CDTI 
within SEVS ego-centric display concepts (R-13). 


47 



MAP 

PLAN 

range 

2.00 

TRK 1320 1 

MAG 


ETA & RTA 


Symbols 

Selected Traffic’s Trend Symbol - Indicates 

Indicating ‘on 

position in 30 seconds and is based on current 

time’ 

# speed. 

1 

Selected Traffic Chevron 



Selected Traffic s Route 

Selected Traffic Intent Symbol - Indication of 
Position required in 30 seconds, to achieve RTA 


Own Ship Intent symbol - Indication of position 
required in 30 seconds to achieve RTA 

Own ship trend symbol - indication of position in 
30 seconds and is based on ships current speed 


Figure 11: 4D Surface CDTI Concepts 

6.3. Integration of CDTI with SEVS - Ego-Centric 

CDTI research has predominately focused on exo-centric presentations of traffic (CDTI) 
and synthetic vision (airport moving maps) on a navigation display; that is, electronic 
moving map concepts. 

However, objective and subjective data from an empirical study suggested that CDTI 
should be presented both head-up and head-down rather than just head-down (Kramer 
and Norman, 2000). Pilot workload for traffic surveillance was lower and it made for 
quicker and easier visual acquisition when the CDTI information was presented both 
head-up and head-down. Traffic locator box icons on the head-up displays allowed for 
quick visual and therefore, much easier detection and assessment of traffic. Clutter was a 
concern but not significantly so. Range filtering of traffic data was shown effective in 
reducing clutter yet retaining critical CDTI information head-up (Wong, Kramer, and 
Norman, 2002). 

Representation of traffic location by icons within an ego-centric perspective display (i.e., 
a “traffic locator box”) creates localization cueing with minimal clutter (Jones, 2002; 
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Kramer and Norman, 2000). Questions emerge as to whether improvement in situation 
awareness can be gained by instead using virtual models of the traffic aircraft, in SV 
presentations. A virtual model, driven from ADS-B ‘In’ information, would theoretically 
provide a more intuitive representation of the traffic and could include aircraft type, 
heading, bank angle, etc. Preliminary evaluation of head- worn display concepts during 
surface operations have evaluated virtual models (see Figure 12) for traffic 
representation, but conclusive results are not yet available. 

Research is needed to identify and quantity the design trade-offs between intuitive traffic 
representation and the potential increases in display clutter or occlusion for ego-centric 
SEVS display concepts (R-14). 



Figure 12: Head-Worn Display Concept With Traffic Represented By A Virtual Model 

Viewpoint considerations must be tailored to the application. Studies in closely-spaced 
parallel approach tasks, suggests that “virtual” viewpoints at other than the pilot eye point 
make a significant difference in traffic awareness and blunder detection (Azuma and 
Furmanski, 2005). In fact, they suggest that “the view that a pilot would normally see out 
of a cockpit was by far the worst option.” 

Research has been conducted to determine minimum relevant traffic information for 
strategic traffic awareness during closely-spaced parallel approach tasks (e.g., Pritchett 
and Hansman, 1996). Others suggest that additional cues (traffic roll and heading) are 
also required to detect traffic blunder (Jennings, Charafeddine, and Powell, 2002). Their 
work suggested that a single synthetic vision display is insufficient to convey traffic 
information. The difficulties arise from an insufficient field-of-view of the PFD (or 
HUD) and that the distance cues are too weak in an ego-centric perspective display to 
precisely convey the location of the other aircraft. While the PFD does show traffic 
elevation and azimuth, additional display concepts have been evaluated to complement 
the SV PFD for traffic awareness, including exo-centric views. 
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7. Concluding Remarks And Recommendations 

NASA LaRC and the FAA are conducting collaborative research activities to ensure 
effective technology development and implementation of regulatory and design guidance 
to support introduction and use of SEYS technologies in NextGen. SEVS technologies 
have the potential to provide an additional margin of safety and aircrew performance to 
enable the implementation of operational improvements for low visibility surface, arrival, 
and departure operations in the terminal environment with equivalent efficiency as visual 
operations. 

In this report, the research literature is reviewed to assess pilot awareness and detection 
of traffic and obstacles when using SVS and EFVS systems. 

Research was reviewed as to how the visual traffic and obstacle detection task is, in 
essence, a human observer performing “target acquisition.” The cognitive and physical 
mechanisms for acquisition and detection using natural vision and EV using head-down 
or head-up displays is discussed. The critical issues influencing the time required, 
accuracy, and pilot workload associated with recognizing and reacting to potential ground 
collisions or conflicts with other aircraft, vehicles and obstructions related to the use of 
SEVS technologies are described. In particular, the importance of sensor resolution and 
the relative contrast of the “target” are paramount. 

This work also discusses the effect of head-down and head-up operations with SV and 
EFYS, respectively, as well as the influence of single and dual pilot operations. The 
pilot’s role in the operation is a significant contributor to the attention directed to and the 
importance of the search task. 

Finally, research is reviewed in the methods and strategies of adding CDTI with head- 
down SVS and head-up EFVS in low- visibility landing and surface operations and their 
effect on time required, accuracy, and pilot workload for recognizing and reacting to 
potential ground collisions or conflicts. 

7.1. Recommendations 

Based on this review, numerous knowledge gaps have been identified and form the 
following recommendations for continued research and also, for specific empirical pilot- 
in-the-loop ground and flight testing activities to support SEVS for NextGen: 

Future work should investigate the vision system technology (SEVS) requirements for an 
equivalent visual operational capability (R-l). 

• To achieve the equivalent visual operations, one could assume that the vision 
system technology must meet the same performance as that provided by aircraft 
windows with a human observer. Is our knowledge of human visual performance 
sufficient and/or correct so as to design and create a vision system which meets 
equivalent visual capabilities? Is this assumption valid? Are these requirements 
too strict? Too lax? 
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• Are head-worn displays required to create this functionality? If so, what are the 
requisite cognitive and engineering issues? If not, what are the requisite cognitive 
and engineering issues with using HUDs and/or HDDs? 

Future work should explore the impact, if any, of SEVS in changing the roles and 
responsibilities of the ANSP ( R-2 '). 

• Will new and expanded “operating credit” from the use of SEVS technologies 
require changes in ANSP roles or responsibilities? What impacts? 

Future work should explore the potential for SEVS to replace the need for or functional 
elements currently required for Surface Movement Guidance and Control Systems and 
for approach lighting systems (R-3). 

• Can equivalent visual capability on the flight deck obviate the need for certain 
requirements currently stipulated for Surface Movement Guidance and Control 
Systems and for approach lighting systems? What are the system characteristics 
required for these credits? 

Research should be conducted to determine the desired/required missed detection and 
false alarm rates for a SV which utilizes ‘behind the glass ’ processing for automatic 
database, obstacle/object, and navigation error and omission checking (R-4). 

• Research suggests that the ‘ideal’ vision system is a SV presentation with 
processing ‘behind-the-glass’ to support integrity and accuracy requirements. 
What level of human interaction with this process is required/desired? What type 
of interaction is required/desired? 

Research should be examine head-down vs. head-up performance for the PM, with 
experimental variations in display size, location, and minification/magnification (R-5) 

• During EFVS operations, the PM has specific roles and responsibilities. Research 
is needed to identify the display characteristics required to best support these roles 
and responsibilities for current and emerging operational capability. 

Research should be conducted evaluating qualitative and quantitative metrics of clutter 
in actual flight conditions (R-6). 

• Display clutter is critical. Qualitative and quantitative metrics have been 
proposed but they have only been tested in ground simulation. 

Research should be conducted using high fidelity simulation and flight testing; evaluating 
HUD attention capture effects in very low visibility operations (visibilities less than 700 
ft) where the saliency becomes less distinct ( R-7 ’). 

• Emerging trends suggest that HUD attention capture could be on the increase. 
Research is needed to develop counter-measures and establish 
guidelines/requirements to ensure that attention capture is mitigated. 


51 


Research should be conducted evaluating the impact of color HUDs (and head-worn 
display or virtual display) on attention capture, clutter, and target detection (R-8). 

• Only a very limited research basis is available regarding the application of color 
virtual displays, head- worn displays, and HUDs. Research is needed in 
“augmented reality” scenarios, especially for attention capture, clutter, and target 
detection, where the interaction of displayed information and real-world visual 
cues are paramount. 

Research should be conducted evaluating the impact of imprecision in conformal HUD 
symbology (statically and dynamically) on attention capture effects and the ability of the 
pilot to use EFVS imagery especially as it is the predominant source of visual 
information for landing in very low visibilities (<700 ft visibility) (R-9). 

• What are the static conformal and dynamic response requirements for an EFVS 
system to conduct operations in very low visibilities where the EFVS provides the 
vast majority, if not the sole source of usable visual cues for the pilot? 

Research should be conducted evaluating the impact of HUD brightness as well as 
simulated OTW visual luminance (apparent target contrast) on attention capture effects 
and the ability of the pilot to detect target (R-10). 

• Data are needed to guide the use and training of the HUD controls to ensure its 
effective operational use. 

Research should be conducted evaluating the impact of experience and training in HUD 
operations for runway incursion detection and attention capture effects (R-ll). 

• The human factors research data on HUD effects do not adequately or 
appropriately reflect trained HUD operators and these influences. Selected data 
should be collected to quantify these impacts. 

Research should be conducted evaluating the use of CDTI for traffic awareness and 
detection on the runway without traffic collision detection indications or alerting. This 
work would be complementary to research on indications, warnings, and alerting 
methodologies to establish the cognitive and human-centered and engineering design 
trade-offs of Terminal Maneuvering Area Conflict, Detection, and Resolution systems (R- 
12 ). 

• SURF-IA and other initiatives are creating significant safety enhancements using 
ADS-B traffic information and attendant CDTI in exo-centric applications. 
However, several gaps in this research and application are apparent, including: 

o Is CDTI on head-down displays sufficient if the pilot’s attention (i.e., roles 
and responsibilities) are directed elsewhere? What display concepts / 
alerting will help this scenario? Display location effects? 

o How does the use of CDTI and instantiations of traffic information 
(including trend information, cleared traffic route, and intent) modulate 
head-down time and attention? 
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o How do pilot roles and responsibilities modulate head-down time and 
attention? 

o Most CDTI studies have assumed “perfect” traffic position reporting. 
What are the influences of realizable/realistic ADS-B accuracies? What 
are the influences of these accuracies on the task and performance? What 
about mixed equipage? ADS-B faults or failures? 

Research should be conducted evaluating the optimal methods for integration of CDTI 
within SEVS ego-centric display concepts (R-13). 

• What methods and presentation formats for ego-centric CDTI are optimal? What 
display concepts / alerting will help this scenario? 

Research is needed to identify and quantity the design trade-offs between intuitive traffic 
representation and the potential increases in display clutter or occlusion for ego-centric 
SEVS display concepts (R-14). 

• Display clutter is critical. Is symbolic representation of traffic on a head-up or 
head-down display sufficient or preferable to actual traffic models? 

• Ego-centric CDTI presentations are likely impacted by “imperfect” ADS-B 
accuracies. What are the influences of these accuracies on the task and 
performance? What methods can be used to compensate for realizable/realistic 
ADS-B accuracies? 
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